When accessing a large number of files continuously, it takes a lot of time to access files and it may affect the performance and operation of the entire system for the following reasons:
It takes time to get new page cache with increase of unused page cache.
If the release process runs to keep the number of i-node cache of the meta-data server below the limit value, the system load increases.
Consider the following corrective actions:
Before and after processing, you can clear all caches for GFS if you unmount target file systems on all nodes and then mount them. By doing this, you can prevent the increase of the processing time due to unused cache.
By dividing the process accessing a large number of files continuously into small steps and setting the processing intervals with sleep(3), the load of the entire system may be reduced and the performance of other processing that runs concurrently may be improved.
If accessing total of 50000 files or more on all nodes continuously, it is recommended to release dentry caches periodically to discard GFS page cache.
Release unused dentry caches according to the following procedure.
Check the usage status of dentry caches.
# /bin/cat /proc/sys/fs/dentry-state <Enter> 411155 391674 45 0 0 0 |
The second field indicates the number of unused dentry caches.
Release unused dentry caches.
# /bin/echo 2 > /proc/sys/vm/drop_caches <Enter> |
Note
Caches other than the GFS file system are released with the command.
Information
You can periodically check and release unused dentry caches by using cron(8).
