CakePHP PHP memory exhausted

[blue-eye-labs]

Hi all,

I've been working on a project that has a fairly complex database and I've been using CakePHP as the framework. After expanding the database (adding some more tables) and then adding the appropriate models, PHP gives me the following:

Fatal error: Allowed memory size of 8388608 bytes exhausted (tried to allocate 16 bytes) in /Users/gid/Sites/primary-energy-research/cake/libs/model/model.php on line 452

Note that the file / line changes each time. This does, of course, mean that almost nothing appears in the logs except PHP telling me (again) of the problem. Is there anything anyone can think of (is my PHP memory limit just too low).
Any ideas on how to debug? I'm running MAMP.

[Jarrod1937]

What is your script doing? A script shouldn't be using more than 8 MiB generally. You can raise your memory limit under the config, but i'd advise only doing so if you know it should be using that much, otherwise there is a severe error in your coding. If its using that much by accident i'd add this function and echo its result at various stages in your script to get an idea where its memory usage is growing and where it fails.
http://php.net/manual/en/function.memory-get-usage.php

That will at least help you isolate the code block that is using so much.

[blue-eye-labs]

That's what I thought. This is normally symptomatic of a loop gone wrong or something. The weirdest thing is that it's happening even on the index page which shouldn't make any database calls (and in which I haven't changed anything).
The problem with using memory_get_usage is that it is helpful to know all the exact orders of function calls within the CakePHP machine as it were. Unfortunately, I don't, and it is a big package.
I will try and work out what changed between commits.

[blue-eye-labs]

Okay, I removed all my models and it seems to be working again. This is strange because the index page shouldn't even be accessing the database.

[blue-eye-labs]

Okay, I have been echoing the memory usage while slowly adding classes and found that the memory was constantly at about 1.0-1.3 MB (but rose as I added more classes despite the fact that they shouldn't be accessed at this point anyway). Then I found a particular class to be causing the problems.

It seems that the relationship information in this class is causing the problem but I can't see why...

[NogDog]

Okay, I have been echoing the memory usage while slowly adding classes and found that the memory was constantly at about 1.0-1.3 MB (but rose as I added more classes despite the fact that they shouldn't be accessed at this point anyway). Then I found a particular class to be causing the problems.

It seems that the relationship information in this class is causing the problem but I can't see why...

When you include/require a class definition, something is going to get added to the script's memory as it reads, parses, and stores that definition, even if you have not yet instantiated any objects from that class (each object presumably needing its own chunk of memory).

As to the particular class that is the problem, does it eat up lots of memory just including its definition, or when you instantiate it, or when you call a particular method from it?

[Mindzai]

I work with some fairly complex cake apps and they never get near 8MB - something is wrong there. What associations are you defining in the offending model?

I would suggest installing XDebug and *cachegrind and profiling the app to see exactly where this is happening at the method-call level.

[blue-eye-labs]

@NogDog:

I didn't include the files, just created them. The class should not have been loaded into the memory at all at this stage. I gather that CakePHP lazily loads classes from their files rather than loading them all at once (which would, let's face it, be pretty stupid).

@Mindzai:
As regards xdebug: I cannot get it to work with MAMP - I did try. As regards associations: I actually removed lots of associations in all my models and left only the ones that I thought it necessary to include. It's possible that maybe something was recursively looping as regards the associations? I don't know.
This has fixed the out of memory problem BUT I now get "the server unexpectedly dropped the connection" more whenever I try and access pages that use the models apart from the user model. Any ideas what is doing this?

[Mindzai]

It's possible that maybe something was recursively looping as regards the associations? I don't know.

It's unlikely, models do recurse quite heavily in cake, but as they are passed by reference, even though variables/properties containing them exist oin many places, they are actually just references to the same instance, so it won't consume more memory. The cake core's factory also ensures a single instance of a model exists at once (unless you bypass the factory and create your own manually). Also PHP internally handles object recursion so it isn't an issue. Are you 100% sure you don't have any queries going on? Do you have debugging turned up to >= 2 in order to check the SQL log? I'm wondering if a model is loading a lot of data and that is consuming your memory.

This has fixed the out of memory problem BUT I now get "the server unexpectedly dropped the connection" more whenever I try and access pages that use the models apart from the user model. Any ideas what is doing this?

That's a new one on me, I've never had that error. Maybe if you post some of your models we might spot something?

[blue-eye-labs]

@Mindzai, thanks good to know about Cake's guts a little.

For some reason enabling xdebug and disabling the Zend optimiser has fixed the previous problem so I can now look at what is going wrong in my code.

I've been checking my scripts and I've found that this part may be the problem. The story goes as follows.
I have a model called Field, each Field MAY have a parent field related to by the foreign key "parent_id".
I have the name of a parent field and would like to fetch all fields whose parent is that field. Following instructions on the CakePHP website (http://book.cakephp.org/view/74/Complex-Find-Conditions) I created the following statement:

$conditionsSubQuery = array('"Parent"."name"' => urldecode($this->params["named"]["field"]));
$dbo = $this->Field->getDataSource();
$subQuery = $dbo->buildStatement(
    array(
        'fields' => array('"Parent"."id"'),
        'table' => $dbo->fullTableName($this->Field),
        'alias' => 'Parent',
        'limit' => null,
        'offset' => null,
        'joins' => array(),
        'conditions' => $conditionsSubQuery,
        'order' => null,
        'group' => null
    ),
    $this->Field);
$subQuery = ' "Field"."parent_id" IN (' . $subQuery . ') ';
$subQueryExpression = $dbo->expression($subQuery);
$conditions[] = $subQueryExpression;
$this->Field->find('all', compact('conditions')); 

Unfortunately I get the following mysql error:

1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '."parent_id" IN (SELECT "Parent"."id" FROM `fields` AS `Parent` WHERE "Parent"' at line 1

the SQL generated is:

SELECT `Field`.`id`, `Field`.`name`, `Field`.`description`, `Field`.`parent_id`, `Parent`.`id`, `Parent`.`name`, `Parent`.`description`, `Parent`.`parent_id` FROM `fields` AS `Field` LEFT JOIN `fields` AS `Parent` ON (`Field`.`parent_id` = `Parent`.`id`) WHERE "Field"."parent_id" IN (SELECT "Parent"."id" FROM `fields` AS `Parent` WHERE "Parent"."name" = 'Test Field' )

I also tried another method:

$this->set("fields", $this->Field->find("all", array(
    "joins"    =>    array(
        array(
            "table"            =>    "fields",
            "alias"            =>    "Parent",
            "type"            =>    "inner",
            "foreignKey"    =>    false,
            "conditions"    =>    array("Parent.name LIKE \"" . urldecode($this->params["named"]["field"]) . "\"")
        ),
        array(
            "table"            =>    "fields",
            "alias"            =>    "Field",
            "type"            =>    "inner",
            "foreignKey"    =>    false,
            "conditions"    =>    array("Field.parent_id = Parent.id")
        )
    )
))); 

Which produces the following SQL and error:

SELECT `Field`.`id`, `Field`.`name`, `Field`.`description`, `Field`.`parent_id`, `Parent`.`id`, `Parent`.`name`, `Parent`.`description`, `Parent`.`parent_id` FROM `fields` AS `Field` inner JOIN fields AS `Parent` ON (`Parent`.`name` LIKE "Test Field") inner JOIN fields AS `Field` ON (`Field`.`parent_id` = `Parent`.`id`) LEFT JOIN `fields` AS `Parent` ON (`Field`.`parent_id` = `Parent`.`id`) WHERE 1 = 1

error:

1066: Not unique table/alias: 'Field'

I'm really just not sure how to achieve this now and it's evidently causing problems...

[Mindzai]

You shouldn't need to do anything like as complicated as that. I would suggest just using the Tree Behavior. With very few exceptions, if you find yourself having to manually join tables or deal directly with the Dbo like this it suggests you have an issue with how your associations are defined. Even without the tree behavior you could define an aliased association back to the same model which would result in a self-join:

public $belongsTo = array(
    'ParentField' => array(
        'className' => 'Field',
        'foreignKey' => 'parent_id'
        // conditions, group, order etc
    )
) 

Liek I say though it sounds like the Tree behavior may be quite suitable here.

[blue-eye-labs]

Thanks for the heads-up on that Mindzai, much appreciated.

[blue-eye-labs]

I used it as follows:

$parentField = $this->Field->findByName(urldecode($this->params["named"]["field"]));
            $this->set("fields", $this->Field->children($parentField["Field"]["id"], true)); 

I would like to do this in one query but it looks like I have to settle for it like this. Am I missing something?

[Mindzai]

No by default the TreeBehavior will only take an id rather than a conditions array - if you were to pass in a conditions array it would be applied to all of the results, ie the parent and the children and your likely wouldn't get a match, or would only get the parent item. I say by default because you could extend the behavior yourself, but all you would be doing is shifting the added query down through an extra layer of abstraction. I wouldn't worry about it too much though, cake's model caching makes for pretty efficient querying.

One thing I would do though is move this into a method of your model. Your controllers should be as sparse as possible. Custom logic such as this should really be placed in the model, then your controller just has to make a single method call and pass the result on the view. That should be about the limit of the controller's involvement.

[blue-eye-labs]

Okay, I'll move the logic into the model, cheers.