1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
|
ORANGEFS
========
OrangeFS is an LGPL userspace scale-out parallel storage system. It is ideal
for large storage problems faced by HPC, BigData, Streaming Video,
Genomics, Bioinformatics.
Orangefs, originally called PVFS, was first developed in 1993 by
Walt Ligon and Eric Blumer as a parallel file system for Parallel
Virtual Machine (PVM) as part of a NASA grant to study the I/O patterns
of parallel programs.
Orangefs features include:
* Distributes file data among multiple file servers
* Supports simultaneous access by multiple clients
* Stores file data and metadata on servers using local file system
and access methods
* Userspace implementation is easy to install and maintain
* Direct MPI support
* Stateless
MAILING LIST
============
http://beowulf-underground.org/mailman/listinfo/pvfs2-users
DOCUMENTATION
=============
http://www.orangefs.org/documentation/
USERSPACE FILESYSTEM SOURCE
===========================
http://www.orangefs.org/download
Orangefs versions prior to 2.9.3 would not be compatible with the
upstream version of the kernel client.
BUILDING THE USERSPACE FILESYSTEM ON A SINGLE SERVER
====================================================
When Orangefs is upstream, "--with-kernel" shouldn't be needed, but
until then the path to where the kernel with the Orangefs kernel client
patch was built is needed to ensure that pvfs2-client-core (the bridge
between kernel space and user space) will build properly. You can omit
--prefix if you don't care that things are sprinkled around in
/usr/local.
./configure --prefix=/opt/ofs --with-kernel=/path/to/orangefs/kernel
make
make install
Create an orangefs config file:
/opt/ofs/bin/pvfs2-genconfig /etc/pvfs2.conf
for "Enter hostnames", use the hostname, don't let it default to
localhost.
create a pvfs2tab file in /etc:
cat /etc/pvfs2tab
tcp://myhostname:3334/orangefs /mymountpoint pvfs2 defaults,noauto 0 0
create the mount point you specified in the tab file if needed:
mkdir /mymountpoint
bootstrap the server:
/opt/ofs/sbin/pvfs2-server /etc/pvfs2.conf -f
start the server:
/opt/osf/sbin/pvfs2-server /etc/pvfs2.conf
Now the server is running. At this point you might like to
prove things are working with:
/opt/osf/bin/pvfs2-ls /mymountpoint
You might not want to enforce selinux, it doesn't seem to matter by
linux 3.11...
If stuff seems to be working, turn on the client core:
/opt/osf/sbin/pvfs2-client -p /opt/osf/sbin/pvfs2-client-core
Mount your filesystem.
mount -t pvfs2 tcp://myhostname:3334/orangefs /mymountpoint
OPTIONS
=======
The following mount options are accepted:
acl
Allow the use of Access Control Lists on files and directories.
intr
Some operations between the kernel client and the user space
filesystem can be interruptible, such as changes in debug levels
and the setting of tunable parameters.
local_lock
Enable posix locking from the perspective of "this" kernel. The
default file_operations lock action is to return ENOSYS. Posix
locking kicks in if the filesystem is mounted with -o local_lock.
Distributed locking is being worked on for the future.
DEBUGGING
=========
If you want the debug (GOSSIP) statments in a particular
source file (inode.c for example) go to syslog:
echo inode > /sys/kernel/debug/orangefs/kernel-debug
No debugging (the default):
echo none > /sys/kernel/debug/orangefs/kernel-debug
Debugging from several source files:
echo inode,dir > /sys/kernel/debug/orangefs/kernel-debug
All debugging:
echo all > /sys/kernel/debug/orangefs/kernel-debug
Get a list of all debugging keywords:
cat /sys/kernel/debug/orangefs/debug-help
|
