Changes of Simple Parallel Exec from #5 to #6

Changed lines at line 7
1 Specification
1.1 Cluster
- Linux machines
- SSH (public key authentication)
- (SCP sequrity copy, optional)
- HTTP 
1.1 Features
- one master computer with server program(Is a HTTP server). SSH and SCP is needed to get the client program to the slave and start it.
- list of slave computers (names or IPs). Every slave acts as an HTTP client. 
- command specification: commandline pattern with space holders for variables and input file generation
- result specification: standart output and/or files
- validation of the results 
- list with task characterised through parameters.
- timeouts and multiply task assigments if necessary (timeout, free resources and so on)
- collecting rules: a) plain concat b) blockwise with parameters
- simple statistiks: which slave did what and which parameter sets failed.
1.1 Error detection
- error while connection/authentication
- machine dead (for whatever reason)
- programm terminated without success
- programm doesn't return within timeout
1.1 Server
- initialisation: for every slave: try to start client (ssh). If it fails: check ssh connection with dummy ssh command. Success -> copy client to slave using scp and try to start it.
- on http request for executeable: reply with the binary
- on http request for new task:  reply with  the next command to execute and all parameters.
- on post: validate result and mark set and collect results
- no more parameter set to process: exit and display statistik.
1.1 Client
- gets via command line: Session ID, command name (path and name of the executeable), md5 checksum of the executeable, server name and port
- check for the executeable: if not there or the MD5 checksum is wrong: fetch it from the server 
- fetch a task
- run program
- check return code: if failed -> Post failture otherwise take the results and post them.
- fetch next task.
- die is there is no more task or the server is not responding.
- different settings for termination: delete executeable (if fetched), delete the client program?
 
1 Implementation Details
1.1 Configuation and Files
- Cluster file: list of computers, one per line
- Parameter file: csv file, cells seperated with |, parameter names in the headline and every following line contains one parameter set. All lines have to have the same amount of cells like the headline!
- Commandline specification: Command line: a perl syntax String with parameter Variables for the parameters; Input files: Name of the file and and parameter name to write in.
1. Terminology:
- Master: is the Computer where the server runs
- Slave: is one of many Computers that do the work
- Server: program that coordinates the process
- Client: program that runs on slaves
- Worker: program that does the computation (any)
- Task: a set of parameters/ settings for the Worker
1 Specification
1.1 Cluster
- One Master 
- Many Slaves 
- primary Linux machines, however the design is platform independent
- any TCP/IP network connection (can be non permanent, but should not :-))
- SSH (public key authentication) on Slave
- SCP sequrity copy (public key authentication) on Slave
- HTTP (any port open on Master, prefered #80)
1.1 Features
- one master computer with server program (HTTP server). SSH and SCP is needed to get the client program to the slaves and start it.
- list of slave computers (names or IPs). Every slave acts as an HTTP client. 
- platform dependent workers possible
- command specification: commandline pattern with space holders for variables and input file generation
- result specification: standart output and/or files
- validation of the results 
- list with task characterised through parameters.
- timeouts and multiply task assigments if necessary (timeout, free resources and so on)
- collecting rules: a) plain concat b) blockwise with parameters
- simple statistiks: which slave did what and which parameter sets failed.
1.1 Error detection/ dealing
- error while connection/authentication (ssh, scp)
- machine dead (for whatever reason)
- programm terminated without success
- programm doesn't return within timeout
- server breaks or gets stopped
1.1 Server
- initialisation: for every slave: try to start client (ssh). If it fails: check ssh connection with dummy ssh command. Success -> copy client to slave using scp and try to start it.
- on http request for configuration: reply client configuration (i.e. what to do on exit)
- on http request for executeable: reply with the binary for the right platform
- on http request for new task:  reply with  the next command to execute and all parameters.
- on post: validate result and mark set and collect results
- no more parameter set to process: exit and display statistics.
1.1 Client
- gets via command line: Session ID, command name (path and name of the executeable), md5 checksum of the executeable, server name and port
- register at the server and fetch configuration
- check for the executeable: if not there or the MD5 checksum is wrong: fetch it (for own platform) from the server 
- fetch a task 
- run program
- check return code: if failed -> Post failture otherwise take the results and post them.
- fetch next task.
- die is there is no more task or the server is not responding.
- different settings for termination: delete executeable (if fetched), delete the client program?
 
1 Protocol
1.1 Configuration
- Request: GET \http://master/config?sessionID=SESSIONID
- Fail (due to wrong session id): 403 (Forbidden)
- Successful Reply: List of Key = Value pairs.
{code:none}
DeleteProgram=Yes/No
DeleteClient=Yes/No
Ping=#
{code}
1.1 Ping (HTTP)
- Ping interval is given in seconds. 0 for no ping.
- Purpose of the ping is that the client realises if the server is stopped or finished or even dead.
- Request: GET \http://master/ping?sessionID=SESSIONID&ticket=TICKET
- Fail due to wrong session id: 403 (Forbidden)
- Successful, but ticket expired (task allready done): 205 Reset Content
- Successful (keep on it!): 204 (No Content)
1.1 Binary
- Request: GET \http://master/binary?sessionID=SESSIONID&name=NAME&platform=PLATFORM
- Fail (due to wrong session id, or wrong name): 403 (Forbidden)
- Fail (due to unsupported platform): 415 Unsupported Media Type
- Success: binary file
1.1 Task
- Request: GET \http://master/task?sessionID=SESSIONID
- Fail (due to wrong session id): 403 (Forbidden)
- Fail (because no task left): 503 Service Unavailable
- Success: 
{code:none}
Ticket=#  (unique number (usually 5 digits))
Timeout=# (in seconds)
CommandLine=commandline
[Input]*
File=filename (or "stdin")
--begin content--
real file content here (binary)
--end content--
[Result]+
Name=resultname
File=filename (or "stdout")
{code}
1.1 Task completed
- Successful: POST \http://master/complete?sessionID=SESSIONID&ticket=TICKET
{code:none}
[Result]+
Name=resultname
--begin content--
file content here (binary)
--end content--
{code}
- Failed: GET \http://master/failed?sessionID=SESSIONID&ticket=TICKET
- Reply Fail due to wrong session id: 403 (Forbidden)
- Reply Otherwise:  200 OK
1 Implementation Details
1.1 Configuation and Files
- Cluster file: list of computers, one per line
- Parameter file: csv file, cells seperated with |, parameter names in the headline and every following line contains one parameter set. All lines have to have the same amount of cells like the headline!
- Commandline specification: Command line: a perl syntax string with parameter Variables for the parameters; Input files: Name of the file and parameter name to write in.