nginx (engine X) – What a Pain in the BUM! [13: Permission denied]

The last 3-4 hours I’ve been struggling with nginx. Everything I did resulted in a “403 Forbidden” error.

Looking at the log files and eventually figuring out how to get more information into said log file I eventually came up with the following that was causing all the problems:

2008/07/22 13:29:31 [error] 3693#0: *1 open() "/var/www/site.com/public/img/bullet_purple.png" failed (13: Permission denied), client: 192.168.1.31, server: site.com, request: "GET /img/bullet_purple.png HTTP/1.1", host: "site.com"

All clever people will simply see this and go “oh that’s a file access permission error, lets see what the permissions are”. The nginx config says to run as nginx:nginx so therefore as long as that group is on the file with a read bit set that process should have access, right? WRONG! Changing the file permissions so it is
-rw-r--r-- 1 site nginx 9665 Jul 22 13:56 bullet_purple.png
does nothing, still permission denied.

If you change the user that nginx is running as to site:site – presto everything works! This seems completely opposite of what is suppose to happen. I can’t be assed looking into the issue though. Right now I just need the files served so I can do some work….

  • danij3l

    i have same error/problem. haven’t been able to resolve it. even in desperate act of madness chmoded entire doc root 777 :D
    than started playing with chown root:root, nginx:nginx, http:http; …..(nginx is started with “nginx:nginx” user).
    any suggestions? (not 2 urgent but VERY annoying)

    sry for my bad english

  • http://nicholasorr.com Nick

    All I can offer you buddy is making sure the permissions match the running user.

    As you have clearly done this, I have nothing. For me this is a test machine and so I’ve left it alone as it is now working. I’ll visit the issue later if I need too :)

    How I got mine working is in nginx.conf I have:

    user  site site;

    Then for directory permissions I have done:

    chown -R site:site /var/www/site.com

  • http://thepinksylphide.com/ Christopher Fritz

    I had this same problem with a site, but somehow is magically went away when I was testing things. Then I added two other sites (the three being virtual hosts). All three sites were created in the same way when I used Apache, and all three are sites running on WordPress. The first one continued to work while the second and third now had the permissions issue. I have a different .conf file for each one in the conf.d folder for nginx, and the two “broken” sites’ config files were made by copying the “working” site’s, then modifying the lines which needed modifying. I diff’d them to ensure they were the same, save for lines with the folder/hostname of the site. That wasn’t the source of the problem.

    By modifying the “user” line in nginx.conf to add the group of a “broken” site, I could get that site working, whereas the other “broken” site remains “broken”, and the already “working” site remains “working”. Restarting nginx is requires for these changes to take effect, so restarting nginx wasn’t the solution, either.

    Playing with permissions got me nowhere. I considered adding user nginx to all the virtual domains’ groups, but that wasn’t a proper solution.

    Running out of ideas, and still no idea how I resolved the issue for the first site only five hours earlier, I decided to reboot the server. Can’t hurt, save for anyone trying to access my main site, right? And after the reboot? Everything worked.

    So, what happened here? I don’t know… My best guess is a file was created setting permissions, and that file remained when restarting nginx. When I rebooted, the file may have been replaced with a new version. If I could go back in time, I would have set nginx’s group to the group of the “working” site, and see if the “broken” sites magically worked.

    To add to the strangeness, html files worked just fine on the “broken” sites. Images and css files did not.

    I hoped I could find a solution other than “reboot” to post here, but maybe someone in my position will read this, and use it as a stepping stone on a path to trying to find a solution beyond “reboot”.

    Even with all these troubles, I’m liking nginx a lot. I moved away from DreamHost to Linode to get away from the slowness of editing more than three posts at a time in WordPress only to find the same problem on Linode. Replacing Apache with nginx solved that problem wonderfully. Now we just need to get enough blog posts and comments about these nginx issues (preferably with solutions!) so users won’t be spending five+ hours like I did today trying to work around a permissions issue…

  • http://nicholasorr.com Nick

    Cheers Christopher :)

    This is exactly why I posted my issue on my blog. Hopefully more and more people will make comments on blog posts and when we are Googling, we are able to find solutions. Nginx is really great, I’m liking it over litespeed now.

    The shortcoming with litespeed is the lack of simple editing of the conf files from a script. I’d have to use an xml library to parse the files or some such. At least with nginx all you need to do is update a text file, or add a text file and the server will pick it up on reboot :)

  • http://nicholasorr.com Nicholas Orr

    Found this thread linking to me with a lots of detailed information maybe you can find info here to help you out :)

    http://www.ruby-forum.com/topic/172018

  • http://tumblr.timakinbo.com/ Tim Akinbo

    You might want to check the permissions also on the parent directories and ensure that they do not restrict the user nginx is running as from accessing the directories.

  • http://nicholasorr.com Nicholas Orr

    I'm thinking this is the case.

    I've been playing more and more with nginx and you do need to make sure every directory starting at / is readable by the user/group that nginx is running as :)

  • Cakes

    The best troubleshooting you can do is enable a shell for the user your running nginx as and find out were in your root path your permission issue is by changing into the directories till you find your perm issue.

  • http://nicholasorr.com Nicholas Orr

    Good point.

    That is what I have been doing actually – as this issue is pretty old. You
    learn things as you go :)

  • http://lupinedev.com Lupinedev

    I know this is old but just fixed this problem today. check in the nginx conf file for the site. Make sure the location is looking for index.php files

    location / {
    index index.php index.html;
    }

  • http://nicholasorr.com Nicholas Orr

    If you don't have a default doc specified you should be able to specify it
    and it should work. I couldn't even do that…

    Thanks for the to though ;)

  • Kyle

    The problem is that nginx requires the whole tree to be readable and not just where your root starts in nginx.conf. So “chmod -R 777 /var” would be the brute force solution.

    The reason running nginx as site fixed it for you is probably because site was the owner for /var.

  • Steve

    The solution here is simple, add execute permissions on your website root directory:
    $chmod +x /path/to/website/root

  • miko

    I know this is old, but I agree with Kyle, although based upon my tests the parent directories need only have the world-executable flag set (i.e. 701 would work). They don't need to be owned by nginx user.

    So if I'm serving files in /var/foo/bar/baz/public, all the parent directories need to have the world-executable flag set or else you'll get permission denied errors.

    Then the files in /var/foo/bar/baz/public need to be world readable (assuming the group owner is not in nginx's group, and nginx is also not the owner of the files in /public). So make them world-readable and nginx will kindly serve them to the public.

    I was just playing around with a static file testing this, so before you go applying to all your sites, you probably want to test more thoroughly, but hopefully this helps out others in the future who run into this error.

  • http://www.gabrielkoen.com/ Gabriel Koen

    Thanks, you saved me from smashing my head against the desk.

  • http://twitter.com/razvandimescu razvan dimescu

    yup, had the same problem. I solved it by changing the user that nginx was runing to the user that created the files from home. You need access starting with root “/”

  • http://icatclaw.com Tower Joo

    Make sure the path tree to be readable by nginx, which will fix the permission issue.

  • bn1

    or you just have to check if you're accessing the right path :D (thanks nginx log) ..I thought, what a jerk I am, but I was really accessing wrong path due missing “rewrite” rule, easing the “/static/” path (it was accessing “/var/www/static/some.site/static/” rather than correct one “/var/www/static/some.site/”)

  • Wguerrero42

    I am having the same issue, but this info is helpful thanks.

  • Saju Sukumar

    Yeah ..that worked…I was breaking my head to solve this issue…just changing the user in conf worked…. (normally its a comment with nobody as username)

    as

    user user_name;

  • Lance Fordham

    I think my scenario is the same as yours. 

    I am running nginx under user “nginx”, and this user is in the group “nginx”. 

    I am running PHP as a Fastcgi process and php is running as user “website”, and this user is in the group “website”. 

    The website files are located in the directory “/home/website” and the “website” user is the owner of this directory.  

    I could execute any PHP file from any subdirectory, but I could not execute/read/write any other file from any other subdirectory other than the /home/website directory because nginx did not have access to those directories and files.

    My solution is to set permissions of the subdirectories that I wish to give nginx access to, to 0711 and set the permissions to all of the files in the subdirectories to 0755.  It works perfectly!

  • Schmoove

    If you change your nginx user and/or group, remember to change permissions to some folders that nginx reads and writes to, like possibly /var/cache/nginx/ or /var/cache/nginx/ — these folders should refect your changes.

  • Michael

    Soooo….. You first say that it's not a permission issue, and then proceed to prove that it is. Was this post supposed to be a joke? Some Linux distributions like to use www as the web user, some like to use www-data, others prefer web, and others have other preferences. Is Nginx supposed to read /etc/passwd file and make its best guess as to the correct user to run as? That would be a pretty dumb idea. Instead, there's a well documented method for changing the user that the Nginx worker processes run as. If you installed Nginx via your package manager, then the people that packaged it for you likely already have that user line in the first line of your nginx.conf.

    The issue has nothing to do with Nginx. As the logs say, you had a permission issue. Anyone with an absolutely basic understanding of Linux could have figured this issue out.

  • http://www.belowtradeprices.co.uk/serve-over-counter.aspx Serve Over Counter

    Hi,

    In fact it has been proved that it's theorically similar to handle connections with threads or with events. Some runtime allow happily to manage hundred of thousand of concurrent threads. Think of the framework/languages they for telecomunication management.

    So one approach or the other is more the opportunity to use an existing good implementation of on way or another than to give a theoretical adventage.

    Number of systems call? Who care. Sorry to say that but show some benchmarks if you want to show that Nginx perform better.

  • John Hess

    This fixed it for me.  I had placed my project in dropbox (don't ask why… I'm not sure I have a good answer) rather than say, my home directory.  By default, everything on the tree had read and execute… except for Dropbox.  

    chmod 755 Dropbox did the trick.

    For anyone who'd be interested, I now plan to move this project out of the dropbox and into a user root so it's deployable to other machines.  I'm using git so the backup is redundant

  • andredieb

    You saved me of several hours of debugging too. Thanks a lot!

  • http://nicholasorr.com/ Nicholas Orr

    I had eventually figured out that the whole tree is what the issue is. As Schmoove has pointed out…

    Even though nginx had access to read this file, it didn’t have permission to access the directory where this file was located.

    One thing that I have come to realise is that information provided from errors are often accurate of the exact issue and not representative of the whole picture.