org.apache.hadoop.security.AccessControlException: Permission denied(org.apache.hadoop.security.AccessControlException: Permission denied)

问题

访问HDFS报错:org.apache.hadoop.security.AccessControlException: Permission denied

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;

public class TestHDFS {
    public static void main(String[] args) throws Exception{
        Configuration conf = new Configuration();
        conf.set("fs.defaultFS", "hdfs://192.168.0.104:9000");
        FileSystem fs = FileSystem.get(conf);

        //存在的情况下会覆盖之前的目录
        boolean success = fs.mkdirs(new Path("/xiaol"));
        System.out.println(success);
    }
}

Exception in thread “main” org.apache.hadoop.security.AccessControlException: Permission denied: user=xiaol, access=WRITE, inode=”/xiaol”:root:supergroup:drwxr-xr-x

解决方案

1.修改配置文件

在hdfs的配置文件中,将dfs.permissions.enabled修改为False

2.更改HDFS根目录权限

hadoop fs -chmod 777 /

3.添加访问用户

hadoop在访问hdfs的时候会进行权限认证,取用户名的过程是这样的:

读取HADOOP_USER_NAME系统环境变量,如果不为空,那么拿它作username,如果为空

读取HADOOP_USER_NAME这个java环境变量,如果为空

从com.sun.security.auth.NTUserPrincipal或者com.sun.security.auth.UnixPrincipal的实例获取username。

如果以上尝试都失败,那么抛出异常LoginException(“Can’t find user name”)

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import java.util.Properties;

public class TestHDFS {
    public static void main(String[] args) throws Exception{
        Properties properties = System.getProperties();
        properties.setProperty("HADOOP_USER_NAME", "root");

        Configuration conf = new Configuration();
        conf.set("fs.defaultFS", "hdfs://192.168.0.104:9000");
        FileSystem fs = FileSystem.get(conf);

        //存在的情况下会覆盖之前的目录
        boolean success = fs.mkdirs(new Path("/xiaol"));
        System.out.println(success);
    }
}

这其实是个漏洞,如果用户使用root或者其他目录的用户名,就可以对相应的目录进行任何操作,所以还是很危险的。 

————————

problem

访问HDFS报错:org.apache.hadoop.security.AccessControlException: Permission denied

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;

public class TestHDFS {
    public static void main(String[] args) throws Exception{
        Configuration conf = new Configuration();
        conf.set("fs.defaultFS", "hdfs://192.168.0.104:9000");
        FileSystem fs = FileSystem.get(conf);

        //存在的情况下会覆盖之前的目录
        boolean success = fs.mkdirs(new Path("/xiaol"));
        System.out.println(success);
    }
}

Exception in thread “main” org.apache.hadoop.security.AccessControlException: Permission denied: user=xiaol, access=WRITE, inode=”/xiaol”:root:supergroup:drwxr-xr-x

Solution

1. Modify the configuration file

In the HDFS configuration file, change dfs.permissions.enabled to false

2. Change HDFS root permission

hadoop fs -chmod 777 /

3. Add access user

Hadoop will perform permission authentication when accessing HDFS. The process of obtaining the user name is as follows:

Read Hadoop_ USER_ Name system environment variable. If it is not empty, take it as username. If it is empty

Read Hadoop_ USER_ Name is a Java environment variable. If it is empty

从com.sun.security.auth.NTUserPrincipal或者com.sun.security.auth.UnixPrincipal的实例获取username。

If all the above attempts fail, an exception loginexception (“can’t find user name”) will be thrown

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import java.util.Properties;

public class TestHDFS {
    public static void main(String[] args) throws Exception{
        Properties properties = System.getProperties();
        properties.setProperty("HADOOP_USER_NAME", "root");

        Configuration conf = new Configuration();
        conf.set("fs.defaultFS", "hdfs://192.168.0.104:9000");
        FileSystem fs = FileSystem.get(conf);

        //存在的情况下会覆盖之前的目录
        boolean success = fs.mkdirs(new Path("/xiaol"));
        System.out.println(success);
    }
}

This is actually a vulnerability. If the user uses the user name of root or other directories, he can perform any operation on the corresponding directories, so it is still very dangerous.