Content and structure of \greed databases
Matevz Tadel
Start date: 8.5.2008
Half-cooked, quarter-written. Need feedback.
Main emphasis, so far, is on:
Determining the content of user-database, its functions and surrounding infrastructure that is required for \greed operation.
Initial thoughts on interface with VOs and their job sources.
Initial thoughts on data-files, file-catalogs, local file caches, data-transfers and file-sharing.
Databases are needed to store \greed state and history.
They are accessed by:
\greedhome and \greedworld servers;
VO's participating in \greedhome;
information servers providing factual and statistical representations of \greed state.
In the following, let us assume that there is a single \greed, with a single set of databases. These databases could actually be distributed in a hierarchical or cloud-like manner, but this should mainly concern internals of the system and not the actual data content, structure and mechanics of change.
\greed-user is a person that participates in the activities of \greedhome or \greedworld. Different avatars that a user can have in \greedworld are not an issue at this point, although the following discussion has some impact on this topic as well. Here we primariliy try to detrmine the attributes of a user that are needed for the operation of \greed.
Unchangeable. GUID or just an integer.
Any pair {account-name, e-mail address} must be unique within
\greed. Thus it might be better to combine them into one entity, as
in account@name@some.domain, and call it account name.
These values can be changed by account admin.
Used to encrypt private communications to the user.
Changed by internals of the \greed. Users consume the credits in \greedworld and can also do transfers to other accounts.
By considering the implications of the above structure, one should notice the following points.
A single person can create several user accounts. One could do that anyway by creating a new e-mail account and registering again into \greed. To not make this too easy, we should: a) charge research credits for opening a new account under the same e-mail; and b) collect monthly fee for each account.
Several certificate-owning identities can acquire the identity of the same user. This should cover the cases of administrative and (small?) group accounts as well as enable users to collaboratively maintain an existing account.
\greed encourages schizophrenia and chaoss.
Nevertheless, the user records should be considered sacred: they will only be changed via authorized means and if the change was admisable by the rules, it is irrevocable. Thus, if certain person misuses the trust of a shared-account admin, this has nothing to do with \greed, it's a personal matter between people sharing the account.
I am not sure about secrecy of these data. It seems reasonable to only expose person/admin DNs and credit status to those, who can login into the account.
Any certificate issued by a CA that \greed trusts can be used. As a fall-back solution, \greed-CA issues its own certificates. The only requirement is to have a working e-mail account that can process encrypted messages. [Is this too severe? Does, say, google-mail support it?]
\greed should also provide jabber service based on account-name. Is it possible: a) to replace authentication mechanism; and b) to have several simultaneous sessions with the same jabber identity?
In principle, the same structure could also be used for machines, services, virtual-machines and virtual machine images. Then, probably, main purpose field should be added to the user struct.
\gled has a well defined concept of identity that can be mapped
directly to the \greed user. See the gled-auth paper.
\gled also supports trivial group-identities - it is just a list of user-identities that can add the group-identity to the list of their active identities.
Should \gled, when running for \grid, use \greed identities exclusively or can it also use other identity sources? External identites would be used for \gled administrators, nodes in given \gled cluster (Saturns) and object-space observation/interaction threads (Eyes).
\gled currently only supports public-private key pair authentication (public keys are stored on master server (Sun Absolute) and challenge is sent to the MIR-emitting-entity (MEE) trying to acquire an identity). The extension to certificates should not be overly complex. Lots of this stuff is now in \root, someone should review that.
What is actually done by \gled on authorization? I'd say:
User sends account id (or name) whose identity he wishes to acquire and a corresponding certificate to \gled.
\gled forwards this request to user-database server (proxy/mirror).
\item User-db server returns a flag saying if this is ok, the public key that should be used to challenge the user and relevant details of the account. This means that root certificates need only be present at the user-db server.
\gled does challenge-response authentication with the user.
User will be asked to sign (encrypt?) all requests for account-data change (mainly spending of credits).
\gled has all identities registered in sun-queen on Sun Absolute. This is a dynamic view of all identities in use in given \gled-cluster, both to represent active MIR-emitting-entities and to satisy MIR-filters attached to active lenses. Expected number of these objects and ways of their management need to be understood, especially when several \gled's use them in the larger context of \greedworld. Very possibly, a special MIR-filter management service will be neessary, so that global changes can be propagated instantaneously.
V0 account status: how many credits can it give away.
VM images
Available job-types: mem/cpu/io consumption, required file-sets, and bonus credit information.
File servers / catalogs.
Predrag proposes to look at BOINC.
Active VOs, available jobs.
Running jobs. Do we care? Transaction by VO that used the resources should be enough. But if we have this info, we can show it to the users and use it to monitor \greed operation.
Here it is really unclear what part should be done by VOs and what by \greed.
Running worlds.
Active MEEs for each account.
Let us first consider types of data-files presen
In principle, \greedhome client.
All other files should be made available by VOs themselves.
If \greed starts to use user CPU for \greedworld simulations, we should provide the VM images.
But, users will be able to contribute also by holding data-files on their disk (and sharing them via bittorrent). So this should be accounted for and file availability checked periodically.
\greedworld client binaries.
Models, textures, sounds, etc.
Basic World data - accessible to everybody.
Detailed world data - result of world exploration. Regional maps, resource availability, detailed data about heavenly bodies (including their trajectories), man-world observation results (e.g, playbacks / images from orbital or aerial surveillance), etc. These data are owned by a user and he can choose to give them to someone alse or to sell them. Are they copiable / sharable? I'd say: a) copiable by the original NPC researcher / lab; and b) yes, sharable.
\greedworld server binaries and credential balls.
True world data. This should be kept private.
MIR backlogs, state snapshots.
To think about: What's in file catalog and where.
To think about:
How clients store files and keep them synchronized.
How these are found by \greedhome, \greedworld and VMs.
Transaction logs.
Job logs. Should VOs do at least some of this?
Various stuff about each account, world, etc. This includes production and mining results and allows us to control we are not being cheated by a hacked world.
Set up \greed CA.
Set up user-db, try accessing it from \root via TSQLXyzz.
I'd go for postgres, via TPgSQLXyzz.
Note that html-docs@root.cern.ch do not have documentation
generated for postgres.
Prepare perl-scripts (or sth) that allow an account admin to change account properties.
Learn about \gled authentication.
Find out how a system service can be started so that it has an active certificate available for use.