Thursday, February 23, 2012

Nginx & PHP-FPM File Upload Battle

As an early starter for this Thursday, I was fighting with Nginx & PHP-FPM to get them accept a little bit larger file uploads. Nginx just aborted the connection after waiting for approximately one minute. However this timeout was not consistent between tries.

After googling for an hour, I was able to found multiple articles / discussions from the web regarding this same question.

Resolving php-fpm ngix 504 Gateway-Timeouts was my starting point. The article didn't help much because I did not even get 504, but I reviewed and tweaked some configuration values mentioned in the article.

I found also a stackoverflow.com question/answer How do I prevent a Gateway Timeout with Nginx? that confirmed that fastcgi_read_timeout affects the timeout between php-fpm and nginx.

php-fpm's php.ini
max_execution_time = 240

php-fpm's pool configuration
request_terminate_timeout = 240s
request_slowlog_timeout = 230s
slowlog = /var/log/php5-fpm.slow.log

nginx.conf
client_header_timeout 240;
client_body_timeout 240;
fastcgi_read_timeout 240; 

Still no luck. From my Apache2 times I recalled that there are also filesize limitations in multiple places, especially in php.ini. I decided to go through them too, but the connection still aborted.

After too many blind trials and errors I decided to check out the log files.

/var/log/nginx/error.log told me the truth:
2012/02/23 04:36:19 [error] 25556#0: *52 client intended to send too large body: 15465840 bytes, client: ...

So nginx refuses the connection because of too large request. Tweaked the nginx.conf:
client_max_body_size 32m;

Excellence! Now the connection didn't abort. But instead, I got an "empty" request to the PHP script: no POST variables nor the uploaded file.

I checked out php-fpm's pool log and found a line like this:
[23-Feb-2012 06:42:28] PHP Warning:  POST Content-Length of 15465845 bytes exceeds the limit of 8388608 bytes in Unknown on line 0

OK. nginx accepted the large body but PHP didn't. After some help from Modwest FAQ: I can't upload a file larger than 8MB through a PHP script I tweaked php.ini with some new lines:

upload_max_filesize = 32M
post_max_size = 32M

And everything works. Why didn't I go through the logs right away?

Summary

php-fpm's php.ini
max_execution_time = 240
upload_max_filesize = 32M
post_max_size = 32M

php-fpm's pool configuration
request_terminate_timeout = 240s
request_slowlog_timeout = 230s
slowlog = /var/log/php5-fpm.slow.log

nginx.conf
client_header_timeout 240;
client_body_timeout 240;
fastcgi_read_timeout 240;
client_max_body_size 32m;

... and enable & read the logs ;)

5 comments:

MTIhai said...

THANK YOU !!!

I was missing some nginx + pool confs, and I was wondering like a mad mad why the upload doesn't work correctly (after setting all the php.ini params like in you post)

Ville Mattila said...

Thanks for commenting! I'm glad my post helped you. ;) -Ville

SimonSimCity said...

Hi, Ville

If I see that right, you're adding more stuff as you need and could improve your security.

Uploaded files are sent in the request-body. Therefore you don't need to exceed the client_header_timeout. 1min, the nginx default, is pretty much, but exceeding it to 4min is not necessary, I think.

request_terminate_timeout is just a backup, if max_execution_time does not work. I would not use that if not really necessary.

Keeping your configuration flexible is of very high value. Therefore I set the configuration for PHP, that can be changed on runtime, in my nginx configuration. Here's an example of changing the file size limitation for upload:


client_max_body_size 5M;
fastcgi_param PHP_VALUE "upload_max_filesize=5M \n post_max_size=5M \n max_input_time=600 \n max_execution_time=600";

I previewed the code and I think you can take it as-is. No characters removed in my comment :)

The benefit of this configuration is, that you can have it per directory and per file. For wordpress you could f.e. just have this config for the following location:

location ~ ^\/wp-admin\/(async-upload|media-new).php$ {
}

Roger Pack said...

Thank you, this fixed a rails upload issue where it wasn't getting to the rails server at all. (I just needed to add the nginx client_max_body_size 32m; to the nginx main body config section). Thanks!

Anonymous said...

Thanks man that helped a lot.

Post a Comment