6.5.2 DFS Setup

Create the mount point and add the DFS entry to /etc/fstab.

Creating the mount point

Create the mount point for mounting the disk partitions on the storage system used as the DFS.

Make the mount point the same as that specified at the master server.

Example

Create the mount point "pdfs" under "/mnt".

# mkdir /mnt/pdfs <Enter>

fstab settings

At "/etc/fstab", define the mount points created above and the logical file system name.

The logical file system name is used to identify the DFS file system. Use the name defined when generating the configuration information of the file system on the master server.

Example

This example shows the mount point "/mnt/pdfs" and the logical file system name "pdfs1" defined at "/etc/fstab".

LABEL=/                 /                       ext3    defaults        1 1
LABEL=/home             /home                   ext3    defaults        1 2
LABEL=/boot             /boot                   ext3    defaults        1 2
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
LABEL=SWAP-sda3         swap                    swap    defaults        0 0

pdfs1                   /mnt/pdfs               pdfs    _netdev	0 0

See

Refer to "6.2.2.1 Creating the Mount Point and Configuring fstab Settings" used when installing slave servers for information about the parameters specified in the fields of the entries added to /etc/fstab.

6.5.2.2 Registering the hadoop Group and mapred User

The hadoop group GID and the mapred user UID must be the same values as the master server hadoop group GID and mapred user UID.

Check the master server "/etc/passwd" file and register the same values.

Example

Check the hadoop group amongst the groups registered at the master server.

# cat /etc/group <Enter>
... omitted ...
hadoop:x:123:hbase
hbase:x:1503: 
bdppgroup:x:1500:

Check the mapred user amongst the users registered at the master server.

# cat /etc/passwd <Enter>
... omitted ...
bdppuser1:x:1500:1500::/home/bdppuser1:/bin/bash
bdppuser2:x:1501:1500::/home/bdppuser2:/bin/bash
mapred:x:202:123:Hadoop MapReduce:/tmp:/bin/bash
hdfs:x:201:123:Hadoop HDFS:/tmp:/bin/bash
hbase:x:203:1503::/tmp:/bin/bash

# useradd -g hadoop -u 202 -c "Hadoop MapReduce" -d /tmp -s /bin/bash mapred <Enter>

Note

If the GID and UID need to be changed to different values because the GID and UID are already registered at the collaboration server, the hadoop group GID and the mapred user UID registered at the master server, slave servers and development server must all be changed to the same values.

6.5.2.3 Mounting

Distribute the DFS configuration information file generated on the master server to the collaboration server to mount the DFS file system.

Copy the DFS configuration information file generated in "6.1.3.11 Generating the DFS File System Configuration Information" for the master server to /etc/pdfs.
```
# scp -p root@master1:dfsConfigurationFileDirectory/client.conf.pdfs1 /etc/pdfs <Enter>
```
Mount the DFS file system.
Example
Mount the DFS file system on the collaboration server.
```
# mount pdfs1 <Enter>
```

Note

The DFS configuration information file must be distributed to the collaboration server before the DFS file system is mounted.
Ensure that the DFS mount sequence is: master servers, slave servers, development servers and collaboration servers. If the slave servers, development servers or collaboration servers are mounted first, then mount will fail, because the master server (MDS) does not exist.

See

Refer to the "mount.pdfs" under "Appendix A Command Reference" in the "Primesoft Distributed File System for Hadoop V1 User's Guide" for details of mounting.

6.5.2 DFS Setup

6.5.2.1 Mount Point Creation and fstab Settings

6.5.2.2 Registering the hadoop Group and mapred User

6.5.2.3 Mounting