Refinement
Zhenhua Guo
Job Submission and Execution. 1
Architecture:
Job Submission and Execution
Query:
(1) Authentication
As we all know, username and password is a fundamental authentication method.
Should we support other types of authentication, especially users’ certificate authentication?
Answer is no. Generally client-side javascript is not allowed to access local file system. Although some systems provide extension to support local file access (e.g. IE ActiveX object: FileSystemObject), this is not portable. As a result, to get user’s certificate, we must provide a text area on the web page where user can paste his/her certificate. You know mostly web pages cope with printable characters. In certificate, there are lots of non-printable characters so that user can not simply copy and paste its content. In a word, lots of tricky work must be done. In the meanwhile, convenience of using out system gets worse because user must do some additional work.
Now let’s rethink over this issue. Why do we need to provide support for certificate authentication? In our system, HTTPS is used as the fundamental transmission protocol. So we don’t worry about eavesdropping attack. By imposing strength constraint on users’ password, we can guarantee desirable security level. So I think username and password authentication is enough for us.
(2) Built-in authentication vs. our own authentication
Tomcat provides built-in support for username and password authentication. However, sometimes it is not flexible to satisfy our requirements.
(3) Work flow support
Do we need to support workflow-level composition? For example, one workflow depends on another workflow or two workflows can be executed parallel.
If answer is yes, does Karajan workflow engine support it?
If not, we must support it in javascript and now we encounter another big issue: javascript does not provide multithread support. I think we can do some workaround here. But the performance and functionality may not satisfy us.
(4) Job manipulation
To manipulate a job (cancel, suspend, resume ...), client should have a valid handle to refer to the job. I am not sure how users can get this handle. I think this handle should be returned when user submits a job. Then when user wants to manipulate the job, he/she needs to send that handle to tell agent which job will be manipulated.
(5) Status checking
Does the user have to be authenticated to do status checking? In other words, do we allow status checking issued by anonymous users?
What mechanisms are used to support status checking?
Ø check on demand
Every time a client sends a status-checking request, status repository sends a similar request to executor... This mechanism guarantees that the client always get accurate and real-time result. However, it may incur unnecessary workload. Even if the status of a certain job does not change, the whole process must be performed still.
In this case, a separate status repository does not benefit us except that it results in a clearer logic representation.
Ø update on demand
Every time status of a job changes, executor will notify status repository. When a user sends a request to get status of his/her jobs, agent server gets result from status repository. However status repository does not need to sends request to executor. By this means, unnecessary communication is eliminated.
I am not sure whether Java CoG provides callback support so that a specified action will be performed if a certain event occurs.
Class Design
Client-side Classes:
Foundation Stones:
1) Workflow
Description:
This class is used to encapsulate functionality related to workflow manipulation. A workflow can contain more than one task.
Now, only Karajan workflow language is supported.
Methods:
addTask(Task); //add a task to workflow description
removeTask(Task) //remove a task from workflow description
setWorkflow(String workflowDescription); //set workflow description directly
toXML() //convert description into a string which can be transferred on the wire.
2) WFStatus
Description:
This class contains status of a workflow.
Methods:
getStatus();
3) Task
Description:
This class represents a detailed task. The task
Methods:
initialize(String description) //For example, initialize(“<transfer source=... destination=...>”);
toXML();
4) WFGraph
Description:
This class contains a list of workflows which will be executed and maintains the relationship among these workflows. The detailed representations maybe vary (queue, stack...).
Methods:
addWorkflow(Workflow);
removeWorkflow(Workflow);
5) Agent
Description:
This class communicates with agent server.
Methods:
getURL(); setURL(); //getter and setter for URL of agent
getPort(); setPort();
Workflow manipulation:
Job manipulation:
6) Submit/cancel/suspend/resume/
Description:
This class submits workflows to remote server.
Methods:
submit(WFGraph, Agent);
submit(Workflow, Agent);
...
Monitor:
7) Monitor
Description:
This class contains functionalities which are needed to monitor status of submitted workflows by a user.
Methods:
getStatus(WFGraph, Workflow, Agent); //get status of a workflow,
getStatus(WFGraph, Agent); //get status of all workflows in a workflow graph
Authentication:
8) User
Description:
This class represents a user in our system. Note: this user account does not necessarily appear in the backend grid service account list.
Methods:
getUsername();
getPassword();
toXML();
9) UPAuth (Username Password authentication)
Description:
This class handles authentication of users.
Methods:
auth(User, Agent); //send authentication request to agent server
Agent side Classes
Agent server sits between client and backend executor. So it communicates with two parties.
Communication with client:
Accept requests from clients. Requests include:
(1) Login
(2) Logout
(3) Register
(4) Job submission
(5) Job status query
(6) Job cancellation
(7) Job suspension
(8) Job resumption
In the meanwhile, agent sends requests to backend executor. Requests include:
(1) Job submission
(2) Job status query
(3) Job cancellation
(4) Job suspension
(5) Job resumption
Note: I assume agent and executor trust each other, which is guaranteed by underlying authentication system.
Executor side classes
Executor receives requests from agent and sends requests to MyProxy server or underlying grid service provider.
Requests from agent:
(1) Job submission
(2) Job status query
(3) Job cancellation
(4) Job suspension
(5) Job resumption
Requests sent to MyProxy (Java CoG is used here):
(1) Fetch a certificate
(2) Renew a certificate?
Requests sent to grid service provider (Java CoG is used here):
(1) Job submission
(2) Job status query
(3) Job cancellation
(4) Job suspension
(5) Job resumption
Posts
