OpenSSI Clusters for Linux For Linux

OpenSSI Clusters for Linux project is a comprehensive clustering solution offering a full, highly available SSI environment for Linux. Goals for OpenSSI Clusters include availability, scalability and manageability, built from standard servers.

Technology pieces include: membership, single root and single init, cluster filesystems and DLM, single process space and process migration, load leveling, single and shared IPC space, device space and networking space, and single management space

Here are some key features of "OpenSSI Clusters for Linux":

1. Membership

� courtesy of the CI project � includes libcluster and the cluster command (part of Cluster Tools)

2. Internode Communication

� courtesy of the CI project

3. Filesystem

� CFS is transparently stacked over any ext3 mount, making it instantly and coherently shared across the cluster � CFS can be used for the root and other filesystems � there is mount enforcement across nodes in the cluster so an NFS mount on any node is automatically done on all nodes; � reopen of files, devices, ipc objects when processes move is supported � CFS supports file record locking and shared writable mapped files (along with all other standard POSIX capabilities � HA-CFS is configurable for the root or other filesystems � Lustre is supported; � openGFS was supported but is not currently � GFS will be supported in OpenSSI 2.x but is not currently supported

4. Process Management

� very complete, including: - clusterwide PIDs - process migration and distributed rexec(), rfork() and migrate() with reopen of files, sockets, pipes, devices, etc. - vprocs - clusterwide signalling, get/setpriority - capabilities - distributed process groups, session, controlling terminal - surrogate origin functionality - no single points of failure (cleanup code to deal with nodedowns) - Load leveling at exec() time and during process execution - clusterwide ptrace() and strace - clusterwide /proc/, ps, top, etc. - threaded processes migrate as a group - process groups can migrate as a group

5. Devices

� there is a clusterwide device model via the devfs code � each node mounts its devfs on /cluster/node#/dev and bind mounts it to /dev so all devices are visible and accessible from all nodes, but by default you see only local devices � a process on any node can open a device on any node � devices are reopened when processes move � rocesses retain a context, even if they move; the context determines which node's devices to access by default � single, clusterwide /dev/pts

6. IPC

� all IPC objects/mechanisms are clusterwide: - pipes - fifos - signalling - message queues - semaphore - shared memory - Unix-domain sockets - Internet-domain sockets � reopen of IPC objects is there for process movement � nodedown handling is there for all IPC objects

7. Clusterwide TCP/IP

� HA-LVS is integrated, with extensions � extension is that port redirection to servers in the cluster is automatic and doesn't have to be managed.

8. Paging/Swapping

� not clusterwide � each node is independent

9. Kernel Data Replication Service

� it is in there (cluster/ssi/clreg) � no subsystems are using it

10. CLVM/EVMS

� not there

11. Shared Storage

� we have tested shared FCAL and use it for HA-CFS

12. HA interconnect

� Bonding is supported

13. DLM

� is integrated with CLMS and is HA

14. Sysadmin

� services architecture has been made clusterwide � ps, top, ipcs are clusterwide by default � localview command can limit view to local node

15. Init, Booting and Run Levels

� system runs with a single init which will failover/restart on another node if the node it is on dies

16. Application Availability

� application monitoring/restart provided by spawndaemon/keepalive � services started by RC on the initnode will automatically restart on a failure of the initnode

17. Timesync

� NTP for now

18. Load Leveling

� for connection load balancing, using HA-LVS � process load leveling is on by default � process load leveling can occur at exec() time or during execution � applications must be registered to load level

19. Packaging/Install

� Have source patch, binary RPMs and CVS source options; � Can also build Debian packages from CVS source; � First node is incremental to a standard Linux install � Other nodes install via netboot, PXEboot, DHCP and simple addnode command;

20. Object Interfaces

� standard interfaces for objects work as expected � no new interfaces for object location or movement except for processes (rexec(), migrate(), and /proc/pid/goto to move a process)

What's New in 2.0.0pre2 Development Release:

� This pre-release fixes a number of bugs affecting the HA-CFS cluster filesystem, VPROC process management, and HA-LVS IP load-balancer. � The kernel no longer uses 4K stacks by default to simplify DRBD-SSI installation without recompiling the kernel.