First COG program

Recently, I wrote a simple JavaCOG program which makes use of the globus middleware installed on gridfarm001. Four kinds of objects in the program are important:

• Task--abstraction of the work you want to execute on remote machines.
• JobSpecification
• Service--local representation of remote service.
• TaskHandler

To appropriately set Service object, other two objects are necessary:

• ServiceContact -- specifies endpoint (host and port) you want to interact with.
• SecurityContext--specifies the credential you will use to do the authentication by server.

Because Service is local representation of remote grid service, provider must be specified to complete the translation between upper abstraction and underlying infrastructure. In my case, the underlying infrastructure is Globus. However I don't know the version of Globus. At first, I used provider GT4. However, the program seemed not to have effect before being terminated antomatically. I added additional debugging and logging statements, but nothing was generated. I debugged it step by step, I found that the program was terminated when the handler submitted the task. Then I tried the command line tool provided by JavaCOG and it worked. So I ensured that some configurations in my program were incorrect. Finally, I found that it was because of the version of Globus. Gridfarm installs GT2. I didn't know that and assumed it was GT4. So I used GT4 as the service provider. Then you know the result... One bad thing is that my program didn't give any useful information before being terminated. No exception could be caught. I don't know why.

JavaCOG Abstraction

Java COG Abstraction

Java COGSpecification. 1

Basic Description. 1

General Abstraction Classes: 2

Additional Abstraction. 3

Basic Description

(from http://wiki.cogkit.org/index.php/Java_CoG_Kit_Abstraction_Guide):

“Every Grid job (remoteexecution, file transfer, file operation) is represented as a Grid task. All job-specific details arerepresented as a task specification.Further, the remote execution and file servers are locally represented as service objects. Every service has a provider attribute associated with it thatsignifies the technology in which the service is implemented. In order toexecute the Grid job, a user needs to create an abstract Grid task andassociate a specification and service to it. The task is then submitted to atask-handler. The task-handler extracts the specification details and dependingon the provider attribute of the service translates them into the protocolspecific constructs expected by the backend service. The task is submitted andexecuted by the handler in an asynchronous mode. Hence, the client need notwait (block) for the task to be completed by the remote service. Instead, oncethe task is submitted to the task-handler, the client is free to continue withother activities and gets asynchronously notified once the task is done(completed or failed).”

General Abstraction Classes:

1          Executable Object:

       AnExecutableObject provides a high-level abstraction for artifacts that can beexecuted on the Grid. It can be specialized as a Grid Task or a TaskGraph.

1.1         Task:

A Task is an atomic unit of execution in cog-abstractions. Itrepresents a generic Grid functionality including remote job execution, filetransfer request, file operation, or information query. It has a uniqueidentity, name, execution status, specification, and set of services (forremote execution).

1.2         TaskGraph

A TaskGraph provides a building block for expressing complexdependencies between tasks. Advanced applications require mechanisms to executeclient-side workflows that process the tasks based on user-defined controldependencies. Hence, the data structure representing the TaskGraph aggregates aset of ExecutableObjects (Tasks and TaskGraphs) and allows the user to definedependencies between these tasks.

2          Specification

Every Grid Taskhas an associated Specification that dictates the objective of the task and theenvironment required to achieve the objective. This class just provides basicattributes which are common to almost all kinds of tasks.

Following three classes are subclasses ofSpecification:

JobSpecification:

The JobSpecification mentions the common parameters needed for theremote job execution independent of the low-level implementation.Implementation-specific parameters can be added to the specification asadditional attributes.

FileTransferSpecification:

    TheFileTransferSpecification provides the commonly used attributes for ?letransfers between Grid resources.

FileOperationSpecification:

The FileOperationSpecification offers the functionality to invokeimportant operations on files hosted on remote Grid resources.

3          Service

Every Grid Taskhas a set of remote Services that support the actual execution of the task. Theservice interface is a local representation of the remote Grid service. Everyservice has a provider attribute that specifies the technology andprovider supported by that service. It also has a service contact and a securitycontext specific to that provider.

4          Handler

Cog-abstractionscontains the TaskHandler and the TaskGraphHandler,to process a Task and a TaskGraph respectively. Once a Task or a TaskGraph issubmitted to the appropriate handler, the handler interacts with the desiredGrid implementation and accomplishes the necessary tasks. The handlers incog-abstractions can be viewed as adaptors that translate the abstract definitionsof a Task and TaskGraph into implementation-specific constructs understood bythe backend Grid services.

5          Status

Every ExecutableObject (Task or TaskGraph) has an associated execution status.

6          Permission

The permissionsinterface provides a means to get and set permissions for a class of usersalong with GridFile and FileResource abstractions.

7          GridFile

A GridFile is anabstract representation of a remote file or directory. It represents the basicproperties of a file such as size, name, modification date, access permissionsetc. This abstraction is a passive information carrier and cannot be used tomodify the properties or contents of a remote resource directly. It can be usedas an input to the file resource abstraction to change access permissions of aremote resource.

Additional Abstraction

If what a userwants to do is manipulate file resources ( read, write, delete, copy …),JavaCOG provides an additional abstraction class which makes the task easier. Inthis case, the user does not need to use task model.

1          FileResource

An alternatemodel of abstraction for file operations, bypassing the task model, isalso available for applications that desire direct interactions with the filehosting Grid servers. A FileResource provides all the necessaryfunctionality to directly invoke remote file operations.

 

CSS

I have been updating my blog since last year. To make the content more readable, I manually edit the html/css source code. I modified the template of my blog to add some css rules which can be used repetitively in my blog posts. This saves lots of duplicate typing work. However, it is still a painful task to edit the html/css source code in google blotspot. No syntax highlight, so small a area where edit takes place, manually html escape ... Of course, I can first edit the html/css code in an external editor (notepad, ultraedit...) and then copy and paste the code. But, what is important to me is the pure content of the blog posts. In other words, the decoration and formatting of the content should take as little time as possible. So, I prefer to use rich text editor in which formatting of text/image... can be controlled conveniently. In addition, the editor must have the ability to convert the rich text into html/css format. In my case, I chose MS Word because I can get it easily. Now, I can edit my document in MS Word which I am familiar with and then convert the document into html. But, one problem pops up now. The converted html document contains css <style></style> section in head section which contains various css rules. I CANNOT copy and paste the document directly into google blogspot because the <style> element CANNOT be defined in bodysection. What I type in the text area(blogspot editing) is finally put into body section of the final web page by google server. So, I thought of changing the css rules into inline mode by using the style attribute.
Originally:

<html>
    <head>
        <style>
        .css{
            color:blue;
        }
        </style>
    </head>
    <body>
        <div class=\"css\">test text</div>
    </body>
</html>

Formatted:

<html>
    <head> </head>
    <body>
        <div style=\"color:blue;\">test text</div>
    </body>
</html>

If all the work is done manually, I believe it is a test of one's patience and carefulness. As a computer major student, I would like it to be done programmatically. I need a CSS parser to parse the css rules and know which element refers to which css rule. After that, the remaining work is just to insert the css rules into corresponding places.
After searching in Google, I found W3 Simple API for CSS and Cypress.
SAC is a standard interface for CSS parser and supposed to work with CSS1, CSS2, CSS3 (currently under development) and other CSS derived languages.
Cypress is an open source CSS parser.
Actually, at first I checked well-known layout engines - Mozilla Gecko, Trident, WebCore(currently WebKit). But I did not find an independent css parsing module in those engines.
Now, I decide to use Cypress to write a simple program to do the work I mentioned above.

Java Cog Setup

Today, I downloaded and installed JavaCog. Detailed Information:
Prerequisite
To use grid services, one must obtain the certificate/credential from administrator of the grid. Generally, public/private key mechanism is used. There are two files:

usercert.pem

This file contains the certificate which is requested by server to authenticate the user.

userkey.pem

This file contains the private key which should be kept secretly by user. Usually, this file is encrypted based on the passphase provided by user so that a attacker cannot get useful information even if he gets this file.

Note: the names of those two files do not matter. What you must do is correctly configure the JavaCOG so that it can find those two files. File cog.properties contains corresponding information. Usercert.pem and userkey.pem are default names for those two files.
Setup Steps:
(1) download JavaCOG 4.1.5
Note: I installed the binary archive. If you prefer to install it from the source code, consult the documentation.
(2) unpack the tarball( .tar.gz or .zip)
Suppose the unpacked files are put into directory COGDIR.
(3) cd COGDIR/bin
(4) run cog-setup(under linux/unix) or cog-setup.bat(under windows)
Then a GUI is displayed to guide you to set up JavaCOG step by step. This GUI is just an auxiliary tool to make the setup easier. Actually, all configuration is written into a file called cog.properties. This file is located under directory $(HOME)/.globus.Suppose username is USER1:

In Linux, $(HOME) is /home/USER1.
In Windows, $(HOME) is C:\Documents and Settings\USER1\ (if operating system is installed into partition C)

(5)Put the certificate of the grid service under directory $(HOME)/.globus/cog-certificates.
(6) Modify the file cog.properties to indicate that the newly added certificate should be trusted by JavaCOG and JavaCOG can use the certificate to authenticate the grid server. Actually, more options can be specified. In my case, those two files are 84ff0685.0 and 84ff0685.signing_policy.
So, related lines in cog.properties are:

cacert=C\:\\Documents and Settings\\gerald\\.globus\\cog-certificates\\84ff0685.0

A sample cog.properties file:

#Java CoG Kit Configuration File
#Mon Oct 01 16:20:20 EDT 2007
usercert=C\:\\Documents and Settings\\gerald\\.globus\\usercert.pem
userkey=C\:\\Documents and Settings\\gerald\\.globus\\userkey.pem
proxy=C\:\\DOCUME~1\\gerald\\LOCALS~1\\Temp\\x509up_u_gerald
cacert=C\:\\Documents and Settings\\gerald\\.globus\\cog-certificates\\44879c16.0,C\:\\Documents and Settings\\gerald\\.globus\\cog-certificates\\1c3f2ca8.0,C\:\\Documents and Settings\\gerald\\.globus\\cog-certificates\\84ff0685.0,C\:\\Documents and Settings\\gerald\\.globus\\cog-certificates\\aaaddcdf.0,C\:\\Documents and Settings\\gerald\\.globus\\cog-certificates\\4a6cd8b1.0,C\:\\Documents and Settings\\gerald\\.globus\\cog-certificates\\5aba75cb.0,C\:\\Documents and Settings\\gerald\\.globus\\cog-certificates\\3deda549.0,C\:\\Documents and Settings\\gerald\\.globus\\cog-certificates\\b89793e4.0,C\:\\Documents and Settings\\gerald\\.globus\\cog-certificates\\9a1da9f9.0,C\:\\Documents and Settings\\gerald\\.globus\\cog-certificates\\2c7969d0.0
ip=xxx.xxx.xxx.xxx

Note: Generally, you want to change the passphase which is used to encrypt/decrypt your private key file. You can use command: grid-change-pass-phase

JavaCOG toolkit
In addition to cog-setup, JavaCOG provides many more tools which makes configuration/execution easier. These tools are under directory bin. Here you can find the documentation about those tools. However, the documentation does not cover all those tools.
Credential Management
If you have several computers, you must save credentials (certificates/private keys) on these different machines. It is a boring work. You can use a credential management server to relieve the burden. Myproxy is a popular credential repository. You can store your credentials in a MyProxy repository and retrieve a proxy credential from the MyProxy repository when needed. The actual work is the user generates a proxy certificate and then uploads it to Myproxy server.

MyProxy commands:

• myproxy-init - Store a proxy credential for later retrieval
  The myproxy-init command uploads a credential to a myproxy-server for later retrieval. In the default mode, the command first prompts for the user's Grid pass phrase (if needed), which is used to create a proxy credential. The command then prompts for a MyProxy pass phrase, which will be required to later retrieve the credential. By default, myproxy-init will create a proxy credential from the user's end-entity credentials at ~/.globus/usercert.pem and ~/.globus/userkey.pem to delegate to the myproxy-server.
• myproxy-store - Store end-entity credential for later retrieval
  Unlike myproxy-init, this command transfers the private key over the network (over a private channel). In the default mode, the command will take the credentials found in ~/.globus/usercert.pem and ~/.globus/userkey.pem and store them in the myproxy-server repository.
• myproxy-logon - Retrieve a credential
  The myproxy-logon command retrieves a credential from the myproxy-server that was previously stored using myproxy-init. In the default mode, the command prompts for the MyProxy pass phrase associated with the credential to be retrieved and stores the retrieved credential in the standard location (/tmp/x509up_u).
• myproxy-retrieve - Retrieve an end-entity credential
  The myproxy-retrieve command retrieves a credential directly from the myproxy-server(8) that was previously stored using myproxy-init(1) or myproxy-store(1). Unlike myproxy-logon(1), this command transfers the private key in the repository over the network (over a private channel). To obtain a proxy credential, we recommend using myproxy-logon(1) instead.
  In the default mode, the command prompts for the pass phrase associated with the credential to be retrieved and stores the retrieved credential in the standard location ( ~/.globus/usercert.pem and ~/.globus/userkey.pem). You could then run grid-proxy-init to create a proxy credential from the retrieved credentials.
  
• myproxy-info - Display information about credentials
  
• myproxy-destroy - Remove a credential from the repository
  
• myproxy-change-pass-phrase - Change a credential's passphrase
  
• myproxy-admin-adduser - Add a new user credential
  
• myproxy-admin-change-pass - Change credential passphrase
  
• myproxy-admin-query - Query repository contents
  
• myproxy-admin-load-credential - Directly load repository
  
• myproxy-server - Store credentials in an online repository
  

COG command:

However, the related command in JavaCOG is

cog-myproxy

This command is similar to the myproxy-init/myproxy-logon/myproxy-store/myproxy-store commands in Myproxy client. In other words, you can store/retrieve credentials(proxy credentials or original certificates/keys).
Some typical examples:

cog-myproxy -h gf1.ucs.indiana.edu put #upload the certificate to the myproxy server.
cog-myproxy -h gf1.ucs.indiana.edu get #download the certificate from myproxy server

Proxy Credential

Every time the service you are interacting with needs your certificate to do authentication, you must input the pass phase. GSI provides a delegation capability: an extension of the standard SSL protocol which reduces the number of times the user must enter his passphrase. If a Grid computation requires that several Grid resources be used (each requiring mutual authentication), or if there is a need to have agents (local or remote) requesting services on behalf of a user, the need to re-enter the user's passphrase can be avoided by creating a proxy.
A proxy consists of a new certificate and a private key. The key pair that is used for the proxy, i.e. the public key embedded in the certificate and the private key, may either be regenerated for each proxy or obtained by other means. The new certificate contains the owner's identity, modified slightly to indicate that it is a proxy. The new certificate is signed by the owner, rather than a CA. The certificate also includes a time notation after which the proxy should no longer be accepted by others. Proxies have limited lifetimes.
When proxies are used, the mutual authentication process differs slightly. The remote party receives not only the proxy's certificate (signed by the owner), but also the owner's certificate. During mutual authentication, the owner's public key (obtained from her certificate) is used to validate the signature on the proxy certificate. The CA's public key is then used to validate the signature on the owner's certificate. This establishes a chain of trust from the CA to the proxy through the owner.

cog-proxy-init

It provides a GUI to generate the proxy certificate. The default name of the proxy file is x509up_u_username which is stored in the temporary directory. (Of course, you can change it by modifying the configuration file.). During the generation, you can specify some options: proxy lifetime, key strength...

If you prefer command line tool to generate the proxy certificate, you should use grid-proxy-init command. Roughly, grid-proxy-init and cog-proxy-init provide the same functionality except the user interface. You can choose either one you like.

grid-proxy-init - Generate a new proxy certificate
grid-proxy-destroy - Destroy the current proxy certificate
grid-proxy-info - Display information obtained from a proxy certificate

GRID introduction

GRID computing is becoming more and more important in both computer science community and theoretic science community. Traditional science fields (physics, biology...) have huge amount of data to be processed so that computer resources owned by a single organization can not satisfy the requirements. So grid computing was come up with to solve the problem.

Currently, most promising grid middleware includes: Globus, GLite, Unicore, Crown, (OpenPBS, LSF).
Condor is also a good distributed computing project which is widely used.
Globus is based on OGSA(Open Grid Services Architecture).

OMII-Europe aims to provide key software componenets for building e-inforastructure. Currently, it focuses on providing common interfaces and integration of major Grid software infrastructures. Its goals include interoperability of gLite/UNICORE/Globus/CROWNgrid.

ProActive is an Open Source Java library (LGPL) for parallel, distributed, and multi-threaded computing, also featuring mobility and security in a uniform framework. With a reduced set of simple primitives, ProActive provides a comprehensive toolkit that simplifies the programming of applications distributed on Local Area Networks (LANs), Clusters, Internet Grids and Peer-to-Peer Intranets.
ProActive deploys applications seamlessly on Local Area Networks (LANs), Wide Area Network (WAN), desktops, clusters, parallel machines, and data-centers, using de facto industry standards such as LSF, PBS, Globus and Unicore, or just ssh. ProActive does not require intrusive installation and enterprise IT infrastructure to be modified.

How to find the RFC document you want

RFC3700 defines the Internet Official Protocol Standards which lists all the RFC documents. By searching in this document, you can find the RFC document you want.

Email - POP3

Post Office Protocol3 is defined in RFC 1939. RFC 2249 defines extension to RFC 1939.
Default port is 110.
Request:
Commands in the POP3 consist of a case-insensitive keyword, possibly followed by one or more arguments. All commands are terminated by a CRLF pair. Keywords and arguments consist of printable ASCII characters. Keywords and arguments are each separated by a single SPACE character. Keywords are three or four characters long. Each argument may be up to 40 characters long.
Response
Responses in the POP3 consist of a status indicator and a keyword possibly followed by additional information. All responses are terminated by a CRLF pair. Responses may be up to 512 characters long, including the terminating CRLF. There are currently two status indicators: positive ("+OK") and negative ("-ERR"). Servers MUST send the "+OK" and "-ERR" in upper case.
Life Cycle
A POP3 session progresses through a number of states during its lifetime. Once the TCP connection has been opened and the POP3 server has sent the greeting, the session enters the AUTHORIZATION state. In this state, the client must identify itself to the POP3 server. Once the client has successfully done this, the server acquires resources associated with the client's maildrop, and the session enters the TRANSACTION state. In this state, the client requests actions on the part of the POP3 server. When the client has issued the QUIT command, the session enters the UPDATE state. In this state, the POP3 server releases any resources acquired during the TRANSACTION state and says goodbye. The TCP connection is then closed.
AUTHORIZATION
Once the TCP connection has been opened by a POP3 client, the POP3 server issues a one line greeting. This can be any positive response. An example might be:
S: +OK POP3 server ready
The POP3 session is now in the AUTHORIZATION state. The client must now identify and authenticate itself to the POP3 server. Two possible mechanisms for doing this are described in this document, the USER and PASS command combination and the APOP command.
TRANSACTION
Commands:

STAT

The POP3 server issues a positive response with a line containing information for the maildrop. This line is called a "drop listing" for that maildrop. The positive response consists of "+OK" followed by a single space, the number of messages in the maildrop, a single space, and the size of the maildrop in octets. This memo makes no requirement on what follows the maildrop size.
Examples:
     C: STAT
     S: +OK 2 320

LIST [msg]

Argument: a message-number (optional), which, if present, may NOT refer to a message marked as deleted. If an argument was given and the POP3 server issues a positive response with a line containing information for that message. This line is called a "scan listing" for that message.
If no argument was given and the POP3 server issues a positive response, then the response given is multi-line. After the initial +OK, for each message in the maildrop, the POP3 server responds with a line containing
information for that message. This line is also called a "scan listing" for that message.
Examples:

C: LIST
S: +OK 2 messages (320 octets)
S: 1 120
S: 2 200
S: .
...
C: LIST 2
S: +OK 2 200
... C: LIST 3
S: -ERR no such message, only 2 messages in maildrop

RETR msg

If the POP3 server issues a positive response, then the response given is multi-line. After the initial +OK, the POP3 server sends the message corresponding to the given message-number, being careful to byte-stuff the termination character (as with all multi-line responses).
Example:
Examples:
C: RETR 1
S: +OK 120 octets
S:
S: .

DELE msg

The POP3 server marks the message as deleted. Any future reference to the message-number associated with the message in a POP3 command generates an error. The POP3 server does not actually delete the message until the POP3 session enters the UPDATE state.
Examples:
C: DELE 1
S: +OK message 1 deleted
...
C: DELE 2
S: -ERR message 2 already deleted

NOOP

The POP3 server does nothing, it merely replies with a positive response.

RSET

If any messages have been marked as deleted by the POP3 server, they are unmarked. The POP3 server then replies with a positive response.

UPDATE
When the client issues the QUIT command from the TRANSACTION state, the POP3 session enters the UPDATE state. (Note that if the client issues the QUIT command from the AUTHORIZATION state, the POP3 session terminates but does NOT enter the UPDATE state.)

QUIT

The POP3 server removes all messages marked as deleted from the maildrop and replies as to the status of this operation.

Optional Commands

TOP msg n

return top n lines in mail msg.

UIDL

It is called a "unique-id listing" for that message. A unique-id listing consists of the message-number of the message, followed by a single space and the unique-id of the message. No information follows the unique-id in the unique-id listing.

USER name

To authenticate using the USER and PASS command combination, the client must first issue the USER command.

APOP name digest.

A POP3 server which implements the APOP command will include a timestamp in its banner greeting. The syntax of the timestamp corresponds to the `msg-id' in [RFC822], and MUST be different each time the POP3 server issues a banner greeting.