Skip to content

OSS PHP Frameworks Unit Testing: General

Jump to bottom Edit New page
ptarjan edited this page Oct 16, 2013 · 15 revisions

As part of our effort towards PHP runtime parity, we are utilizing the unit tests of many major open source PHP frameworks to determine where HHVM is lacking functionality or failing with current functionality.

Below is a list of the frameworks we are currently testing. Here is a common set of infrastructure and setup instructions that was used as we conducted the unit tests:

Development Environment

The unit tests are being run on a CentOS 6.3 development machine with Dual 8-core Intel Xeon E5-2660 2.2 GHz processors and 144 GB RAM.

HHVM

The unit tests are always being run with the latest internal build of HHVM. To ensure that you are working with the closest version of HHVM to our internal build, use the instructions for building HHVM from the source on CentOS 6.3.

Composer

Many of the frameworks tests require additional dependencies beyond just the framework source in order to run the tests successfully. Composer is used to retrieve the dependencies since the frameworks provide a composer.json file specifying what needs to be downloaded. To install and use Composer with HHVM, follow these steps:

  1. Download the latest composer.phar to your development machine.

  2. Place composer.phar in a directory of your choosing (e.g., ~/bin). It may be helpful to have the directory where the composer.phar is located as part of your $PATH.

  3. To download the dependencies for any given framework, run the following command from the directory containing the composer.json file:

    hhvm [path-to-composer-phar]/composer.phar install --dev

  4. It may be helpful to create an executable "shortcut" script to run Compose when installing the dependencies. For example, create a script called composer-install-hhvm that has the above command as its contents.

PHPUnit

Note: This section is for installing PHPUnit to run the framework unit tests. These are not the instructions for downloading the PHPUnit source code for the testing of itself. To test the actual PHPUnit framework, see here.

PHPUnit is the primary unit testing tool for the frameworks (although, it is important to note that not all frameworks use PHPUnit as their testing framework - these will be called out specifically when describing the testing for that framework). To mimic the way we installed and used PHPUnit, follow these steps:

  1. Create a composer.json file in a directory where you want PHPUnit to be installed. The contents of the file should be:

    {
  "require-dev": {
  "phpunit/phpunit": "3.7.*",
  "phpunit/php-invoker": "1.1.3",
  "phpunit/dbunit": "1.2.3",
  "phpunit/phpunit-selenium": "1.3.2",
  "phpunit/phpunit-story": "1.0.2"
  }
}

  2. Run Composer in the directory where the composer.json file is located using the method described in the Composer section above: hhvm [path-to-composer-phar]/composer.phar install --dev

  3. A vendor directory will be created. In this directory, you will notice a bin directory. The phpunit "binary" is located in there.

  4. To run PHPUnit:

    hhvm [path]/vendor/bin/phpunit [optional arguments]

  5. It may be helpful to create an executable "shortcut" script to run PHPUnit. For example, create a script called phpunit-hhvm that has as its contents:

    hhvm [path]/vendor/bin/phpunit "$@"

  6. Finally, it may also be helpful to put the vendor/bin directory in your path.

Optional Install Method

You may also try to follow this process to use PHPUnit as well; however, it is possible that different results could occur from the above method since the .phar file does not have all the optional packages that may be necessary to run the unit tests (e.g. phpunit-selenium):

  1. Download the latest phpunit.phar to your development machine

  2. Follow the phar based installation instructions. For our testing, we just placed the phpunit.phar into a directory (e.g., ~/bin) and ran PHPUnit directly from the phar. It may be helpful to have the directory where the composer.phar is located as part of your $PATH.

  3. To run PHPUnit, use the following command line sequence from the directory specified in the particular framework's instructions below (usually the directory containing a phpunit.xml or phpunit.xml.dist file):

    hhvm [path-to-phpunit-phar]/phpunit.phar [optional arguments]

  4. It may be helpful to create an executable "shortcut" script to run PHPUnit. For example, create a script called phpunit-hhvm that has as its contents:

    hhvm [path-to-phpunit-phar]/phpunit.phar "$@"

PHP 5.5.x

When running the unit tests with HHVM, it is sometimes helpful to compare the output with the PHP runtime. Our comparison is against PHP 5.5.3. Here are the steps that we used to setup and install Zend:

  1. Ensure that you have any needed prerequisites for PHP installed on your development box. For example, the following packages should probably be installed.

    curl: sudo yum install curl-devel
    mcrypt: sudo yum install libmcrypt libmcrypt-devel
    autoconf: sudo yum install autoconf

  2. Download the PHP source code. We downloaded PHP 5.5.3 at the time; the latest version may be a bit higher but shouldn't affect the install or the running of the tests.

  3. Extract the PHP source tar file

    tar xzvf [PHP tar file]

  4. From within the extracted directory, configure PHP for compilation:

     ./configure --prefix=/path/to/install/php55 --with-mysql --with-pdo-mysql --with-curl \
 --with-openssl --with-mcrypt --enable-mbstring --enable-pcntl --with-mysqli --enable-intl \
 --with-pdo-pgsql --with-pgsql --with-libxml-dir --enable-opcache --enable-calendar \
 --enable-sockets --enable-zip --enable-soap --with-zlib --with-gd --with-bz2

  5. Now install PHP

    % make
    % make install
    % cp php.ini-development /path/to/install/php55/lib/php.ini

  6. Make some php.ini changes

    opcache: See the section marked "Recommended php.ini settings"
    memory_limit: We changed the max memory limit to 512M
    date.timezone: Change the timezone to your particular location (e.g., America/Los_Angeles). This will avoid warnings like this:

     Warning: strtotime(): It is not safe to rely on the system's timezone settings. 
 Please use the date.timezone setting, the TZ environment variable or the 
 date_default_timezone_set() function. In case you used any of those methods and 
 you are still getting this warning, you most likely misspelled the timezone identifier.

  7. If you want XDebug, follow these steps:

     % wget http://pecl.php.net/get/xdebug-2.2.3.tgz
 % tar -xvzf xdebug-2.2.3.tgz
 % cd xdebug-2.2.3
 % /path/to/install/php55/bin/phpize
 % ./configure --enable-xdebug --with-php-config=/path/to/install/php55/bin/php-config
 % make
 % make install

    In php.ini add the following line:

    zend_extension="/path/to/install/php55/extensions/no-debug-non-zts-20121212/xdebug.so"

Here are some useful links to help with the PHP 5.5.x install above:

http://www.thegeekstuff.com/2008/07/instruction-guide-to-install-php5-from-source-on-linux/
http://www.php.net/manual/en/opcache.installation.php
http://xdebug.org/docs/install
http://pecl.php.net/package/xdebug
http://www.php.net/manual/en/install.pecl.static.php

Dealing with Fatals

During the course of our unit testing, we ran into many instances where HHVM would fatal on a given test and the unit tests would halt. In order to allow the unit tests to continue, we added the following piece of code to the top of any test that was failing:

$this->fail("HHVM Fatals");

The Frameworks

Add a custom footer
Add a custom sidebar
Clone this wiki locally