OSS PHP Frameworks Unit Testing: Symfony

Symfony is an open source web-development framework for PHP.

Source Download

Download the latest master branch from Github

Official Testing Documentation

The documentation to run the unit tests are found here: http://symfony.com/doc/master/contributing/code/tests.html

We generally followed the instructions for running the tests given in the documentation above. Below are some additional details on what we did to get the tests running correctly.

Initial Setup

In order to run the Symfony unit tests successfully, make sure you have HHVM, Composer and other basic packages ready to go.

Some initial configuration is required for Symfony, particularly around dependencies.

Dependencies

We used composer to install the dependencies. In the root Symfony directory, you will find the composer.json file that will be used to download the required dependencies to run the unit tests.

hhvm [path-to-composer-phar]/composer.phar install --dev

Note: We ran into an issue downloading the dependencies with HHVM, particularly with the phing depedency. We believe this is probably due to a proxy problem, but if you run into this type of issue:

[Composer\Downloader\TransportException]
  The "https://api.github.com/repos/phingofficial/phing/zipball/5500c5012520203523fd4ea05dabf1f51c331267" file could not be downloaded:
   allow_url_fopen must be enabled in php.ini (Failed to open https://api.github.com/repos/phingofficial/phing/zipball/5500c50125202035
  23fd4ea05dabf1f51c331267 (Operation timed out after 5000 milliseconds with 1015593 out of 8894907 bytes received))

use PHP 5.5 to download the dependencies.

Running the Tests

We used our phpunit binary to run the unit tests. To run the Symfony unit tests, use the following command from the root Symfony directory:

`hhvm [path-to-vendor-parent]/vendor/bin/phpunit`

Fatals

We ran into quite a few fatals with Symfony. We used our "get around fatals so we can continue testing" approach to make sure we could get through the entire suite of tests.

Reflection and Abstract Methods

We ran into many fatals with the following form:

HipHop Fatal error: Class Mock_Extension_4c0c4e6f contains abstract method (load) and must therefore be declared abstract or implement the remaining methods

We narrowed this down to an issue with code in PHPUnit, actually. For some reason, our implementation of ReflectionMethod is not making some methods abstract properly.

The files affected with these fatals were:

src/Symfony/Component/DependencyInjection/Tests/Extension/ExtensionTest.php (2 tests)
src/Symfony/Component/HttpKernel/Tests/Fragment/RoutableFragmentRendererTest.php (3 tests)
src/Symfony/Component/HttpKernel/Tests/Profiler/FileProfilerStorageTest.php (2 tests)
src/Symfony/Component/Intl/Tests/ResourceBundle/Reader/AbstractBundleReaderTest.php (`setup` METHOD)
src/Symfony/Component/Routing/Tests/Matcher/RedirectableUrlMatcherTest.php (3 tests)
src/Symfony/Component/Security/Tests/Core/Authentication/Token/AbstractTokenTest.php (9 tests)
src/Symfony/Component/Templating/Tests/DelegatingEngineTest.php (1 test)
src/Symfony/Component/Templating/Tests/PHPEngineTest.php (3 tests)
src/Symfony/Component/Templating/Tests/TimedPHPEngineTest.php (1 test)
src/Symfony/Bundle/FrameworkBundle/Tests/Validator/ConstraintValidatorFactoryTest.php (1 test)
src/Symfony/Bundle/SecurityBundle/Tests/DependencyInjection/Security/Factory/AbstractFactoryTest.php (3 tests)

DOMTest::hasAttribute

We received this type of fatal:

HipHop Fatal error: Call to undefined method DOMText::hasAttribute from context Symfony\Component\DomCrawler\Form in /data/users/joelm/php-open-source-projects/symfony/src/Symfony/Component/DomCrawler/Form.php on line 389

or

HipHop Fatal error: cannot create attribute on this node in /data/users/joelm/php-open-source-projects/symfony/src/Symfony/Component/DependencyInjection/SimpleXMLElement.php on line 49

HHVM thinks node is a DOMText which DOES NOT have an has Attribute method.

The files affected with this particular fatals were:

src/Symfony/Component/DependencyInjection/Tests/Extension/ExtensionTest.php (1 Test)
src/Symfony/Component/DomCrawler/Tests/CrawlerTest.php (1 Test)
src/Symfony/Component/DomCrawler/Tests/FormTest.php (1 Test)

Arrays and Traversable

In HHVM, arrays implement the Traversable interface. This is not the case in the PHP runtime.

HipHop Fatal error: Argument must implement interface Traversable in /data/users/joelm/php-open-source-projects/symfony/src/Symfony/Component/Filesystem/Filesystem.php on line 132 

The files affected with this particular fatals were:

src/Symfony/Component/Filesystem/Tests/FilesystemTest.php:testRemoveCleansArrayOfFilesAndDirectories()
src/Symfony/Component/Filesystem/Filesystem.php:remove() and toIterator()

str_getcsv

HipHop Fatal error: Undefined function: Symfony\Component\HttpKernel\Profiler\str_getcsv in /data/users/joelm/php-open-source-projects/symfony/src/Symfony/Component/HttpKernel/Profiler/FileProfilerStorage.php on line 64

We had not implemented this function. It is implemented now however.

File-Based Descriptors

We ran into a problem with file-based descriptors.

HipHop Fatal error: Unexpected object type Unknown. in /data/users/joelm/php-open-source-projects/symfony/src/Symfony/Component/Process/Process.php on line 253

Here is some test code that demonstrates this issue:

<?php
abstract class AbstractProcessTest {
  public function testTTYCommand() {
    $process = $this->getProcess('echo "foo" >> /dev/null');
    $process->setTTY(true);
    var_dump($process);
    $process->run();
  }

  abstract protected function getProcess($commandline);
}

class MyProcessTest extends AbstractProcessTest {
  protected function getProcess($commandline) {
    $process = new MyProcess($commandline);
    return $process;
  }
}

class MyProcess {
  private $commandline;
  private $tty;
  private $pipes;

  public function __construct($commandline) {
    $this->commandline = $commandline;
  }

  public function setTTY($tty) {
    $this->tty = (Boolean) $tty;
    return $this;
  }

  public function run($callback = null) {
    $this->start($callback);
  }

  public function start($callback = null) {
    $descriptors = $this->getDescriptors();
    var_dump($descriptors);
    $proc = proc_open($this->commandline, $descriptors, $this->pipes);
    foreach ($this->pipes as $pipe) {
      var_dump($pipe);
      stream_set_blocking($pipe, false);
    }
  }

  public function getDescriptors() {
    if ($this->tty) {
      $descriptors = array(
        array('file', '/dev/tty', 'r'),
        array('file', '/dev/tty', 'w'),
        array('file', '/dev/tty', 'w'),
      );
    } else {
      $descriptors = array(
        array('pipe', 'r'), // stdin
        array('pipe', 'w'), // stdout
        array('pipe', 'w'), // stderr
      );
    }
    return $descriptors;
  }

}

$mpt = new MyProcessTest();
$mpt->testTTYCommand();

The files affected with this particular fatals were:

src/Symfony/Component/Process/Process.php::start()
src/Symfony/Component/Process/Tests/AbstractProcessTest.php::testTTY()

Includes

We ran into the following type of fatal:

HipHop Fatal error: Class undefined: WithoutRoutesUrlGenerator in /data/users/joelm/php-open-source-projects/symfony/src/Symfony/Component/Routing/Tests/Generator/Dumper/PhpGeneratorDumperTest.php on line 86

After some thinking that this might be a problem with file_put_contents, it was determined that HHVM had an issue with including a file twice to the same path.

Here is some test code:

<?php
$tmpFilePath = sys_get_temp_dir().'/'.'phpgentest.php';
@unlink($tmpFilepath);
file_put_contents($tmpFilePath, '<?php class MyTest{}');
include($tmpFilePath);
file_put_contents($tmpFilePath, '<?php class GreatTest{}');
include($tmpFilePath);
var_dump(get_included_files());
$x = new GreatTest();
@unlink($tmpFilePath);

The Symfony files affected with this fatal were:

src/Symfony/Component/Routing/Tests/Generator/Dumper/PhpGeneratorDumperTest.php 

(each test function in this file has a file_put_contents() on the same path). You run one test individually, fine. All at once, you get the fatal.

Failures and Errors

After getting through the fatals, we will then start to examine errors and failures in functionality.

Call To Action

We need to fix all of the fatals. We have started work and made a bit of progress (e.g., implementing str_getcsv), but there is more work to do.