The ECAI/TimeMap Metadata Clearinghouse
SoftwareUser Guide Vsn 1.0 (1998)
Ian Johnson
Archaeological Computing Laboratory
School of Archaeology
University of Sydney NSW 2006
NOTE: The web-based software has been substantially revised since this manual was written. However, the changes are towards simplification and ease of use, so this manual should prove adequate to explain the software pending revision.
ECAI metadata draft standard: March 1999: HTML document
This document details the modified Dublin Core metadata standard adopted by ECAI
Preamble
The Electronic Cultural Atlas Initiative (ECAI) seeks to encourage the creation and dissemination of electronic datasets of cultural information. One way in which it pursues this goal is through building a searchable index of available datasets - the ECAI Metadata Clearinghouse - which is the subject of this document.
The term dataset is used to describe any set of electronically available information, including:
Further information
Relevant web sites for further information on the ECAI metadata project:
Promotion of the creation and dissemination of cultural datasets in digital map form; regional and thematic electronic cultural atlases.
Methodologies for recording and mapping cultural features which change through time and for accessing such data across the Internet.
ECAI Metadata
Metadata is "data about data", in other words descriptive or cataloguing information about datasets. The aim of metadata is to allow one to search for datasets relevant to a particular problem or enquiry, without having to search the individual datasets. It is the equivalent of catalogue cards in a library - each card points to a specific book and the catalogue allows searches based on title, author or subject keywords (generally through an on-line database).
ECAI metadata is data which catalogues the content, spatial and temporal extent of datasets generated by ECAI associates and others. ECAI metadata is stored in an Internet-searchable database known as the ECAI metadata clearinghouse. The clearinghouse has a number of functions outlined in the next section.
Anyone with a web browser can search the ECAI metadata clearinghouse database. Searches can be applied to all elements or to a specified set of elements (e.g. to search only the title, or by time or geographical location).
Anyone with a web browser can also add data to the clearinghouse. Data entered by the ECAI core teams and by ECAI associates are identified by special markers - searches can be restricted to such data alone.
ECAI metadata is based on the well-established Dublin Core metadata standards. We will refer to it as descriptive metadata to distinguish it from the more technical connection and translation metadata defined by the TimeMap project.
An example of ECAI metadata
The Role Of The ECAI Metadata Clearinghouse
1. Metadata definition control
The ECAI clearinghouse maintains a set of tightly defined metadata definitions appropriate to ECAI, based on the Dublin Core metadata standards. The clearinghouse defines:
Metadata tools
Most metadata entry tools provide a template for data entry but do not control what you put in it. Very often you must refer to a separate set of definitions to know what is expected in each element.
The TimeMap project has written metadata editors which read the metadata definitions from the clearinghouse and provide user-friendly metadata entry under tight control. Elements, schemes and, where appropriate, values are chosen from pulldown lists which ensure rapid and consistent entry of data.
The editors also insert the default set of metadata elements for each new dataset catalogued, greatly speeding preparation of data.
Two metadata editors are available:
http://www.ecai.org/metadata/login.html
Although web pages do not allow the same degree of interaction as purpose-built Windows or MacOS programs, this editor provides tight control of data entry, with pulldown lists of elements, schemes and values, using standard HTML forms and Javascript.
This editor provides greater flexibility and ease of use, but only runs under the Windows operating system (it has not yet been tested on a Power Mac/SoftWindows). Unlike the web-based version it can create TimeMap® translation metadata, allowing SQL server databases to serve geographic data to the TimeMap viewer program (TMView - see TimeMap web site).
2. Centralised storage and searching of metadata
The ECAI clearinghouse stores metadata for all datasets registered with ECAI. Centralised storage ha several advantages:
Note: All datasets registered with the clearinghouse must belong to a specific user of the clearinghouse. Individuals can freely register themselves as users, but all datasets added to the clearinghouse are vetted by ECAI staff to determine their appropriateness and ECAI "rating" before being marked as available for searches.
ECAI ratings
Metadata stored in the ECAI clearinghouse are attributed an ECAI rating based on assessment by ECAI staff. The rating identifies the quality and importance of the datasets registered. The rating system below is highly provisional and is to be discussed and fleshed out at the January 1999 ECAI meeting in Taipei:
3. Indexing ECAI TimeMap datasets
The TimeMap project (University of Sydney) has developed a map-based viewer (TMView) for cultural datasets with a temporal component. TMView can superimpose data from diverse datasets onto a common map, and access suitably formatted datasets on SQL servers across the Internet.
In addition to standard metadata, the ECAI clearinghouse stores the translation information necessary for TMView to connect to, query and display data from Internet-accessible datasets which conform with the TimeMap formatting standards (see TimeMap web site).
The Windows-based metadata editor (TM_MDE) allows the entry of full connection information (server name, address, database name etc.) and the creation of the special TimeMap translation table which allows TMView to connect to and query a remote dataset.
Example TimeMap translation metadata
The metadata illustrated here is a small part of the translation metadata defined in the TimeMap formatting standards. For complete information, consult the TimeMap technical documentation (in preparation December 1998). All TimeMap translation metadata elements start with tm.
In this example, tm.ds_alias provides a short name used by the TMView software to refer to the dataset, tm-dbms_type specifies that the data are stored in an Interbase server, tm.dbms_version specifies that it is Interbase version 4.0 and tm.instance_table_name specifies that the data for feature instances are stored in a table called nsw_heritage_inst
TimeMap enabling of datasets
In order for TMView to connect remotely to a dataset, all that is required is to:
Web-Based Metadata Editor
( http://www.archaeology.usyd.edu.au/research/ecai/metadata/login.html )
The TimeMap web-based metadata editor allows editing of metadata under tight control with any standard web browser supporting JavaScript (Netscape or Internet Explorer Vsn 3 and above - Vsn 4 preferred).
This editor creates ECAI descriptive metadata (based on Dublin Core), and is suitable for registering electronic datasets in any format - CD-ROM, download files, web sites, database-driven pages, SQL servers. However, if registering a TimeMap-enabled dataset on an SQL server, you should use the Windows-based editor (TM_MDE, see page 17) which has additional functionality.
This screen allows brute-force searching of the metadata clearinghouse. Datasets are listed if any of the word(s) typed appear in any of the metadata entries for that dataset. This is the same approach as typing words into a standard web search engine. The clearinghouse can be searched more effectively by following the link to the Power Search (next page) which allows more restrictive searching including searches by geographic location and time period.
To create metadata and register a new dataset with the clearinghouse, or to edit existing metadata, follow the link to Registration Page (next section).
For an explanation of the metadata clearinghouse (this document), plus many links to web sites about metadata and the detailed specifications of the ECAI metadata standard, follow the link What is a Metadata Clearinghouse?
Clearinghouse power search
The Power Search page gives several options for refining the search:
Geographic & temporal search: Geographic searches must be specified in latitude and longitude, with negative values west of Greenwich (-180 to 0) and south of the equator (-90 to 0). Temporal search can use CE (Common Era or AD), BCE (= BC) or BP (Before Present, set by C14 dating conventions at 1950).
Metadata element definitions: The bottom section of the page allows viewing of the definitions of each of the metadata elements, or of metadata sub-elements descended from a particular element (e.g. all elements descended from dc.coverage).
Search results
Datasets satisfying the search will be listed as a set of hyperlinks:
Clicking on a hyperlink displays the data recorded for the dataset:
Registering with the clearinghouse
Before editing metadata you must log in as an existing user, or else create a new login (click New User on the login screen - next page - to register as a new user). You will only have access to metadata for datasets which you have registered using your login.
To use the registration and metadata editing functions you must have JavaScript and cookies enabled in your browser (requires Netscape or Internet Explorer version 3 or above).
Creating a new login
This screen creates a new user of the clearinghouse database. It is essential to record your password and login name, as we currently have no convenient means of finding them out if you lose them. Please avoid creating multiple identities as, at present, we have no way of combining them or changing the ownership of individual datasets.
Neither user name nor password is case sensitive. For security reasons, passwords cannot be overly simple - for example, kdty is too short, kdty482 is too simple (a ‘word’ followed by a number), whereas boo45klet is acceptable (digits embedded in a word, or vice versa).
Logging in
Once you have established a login identity, you can proceed to registering datasets and updating metadata in the clearinghouse. To do so you must identify yourself each time you access the clearinghouse, through the following login screen:
Once entered, your identity is maintained by a ‘cookie’ on your computer which persists until you close your web browser. Name and password are not case sensitive.
Registered datasets
The clearinghouse identifies all the datasets you have previously registered under your login identity and displays them in a list of hyperlinks:
Click on a hyperlink to display and/or edit the connection data and ECAI descriptive metadata recorded for the dataset (page 12).
Click on Register a New Dataset to record connection information and create metadata for a previously unregistered dataset.
Registering a new dataset
Connection information
Clicking on Register a New Dataset creates a new entry in the metadata clearinghouse and brings up the connection information screen (page 13) for entry of information on how to locate the dataset (URL, server details, database name etc.). Fill in the connection information, following the instructions displayed on the screen (page 13), and then click Save Changes.
Descriptive metadata
After a short delay, during which the dataset is registered with the clearinghouse, the default set of ECAI metadata entries will be displayed in the metadata editing screen (page 14) so that you can enter values describing the newly registered dataset.
If you do not edit the metadata for the newly-registered dataset, there is no information in the clearinghouse which would allow this dataset to be retrieved in a search. It would be like entering the title of a book in a library catalogue but entering no author, date, publication details or subject keywords.
In general, you should fill in appropriate values for all the entries in the default metadata set. Some are required, as shown on the editing screen by an asterisk. Most elements, with the exception of DC.Coverage …, may be repeated as often as you like.
If you fail to fill in one of the required elements, repeat one of the non-repeatable elements, or enter numeric values outside the allowable range for a particular entry, you will get a warning message when you click Commit Changes.
See discussion of the metadata editing screen (page 14) for further information.
Viewing metadata as the registered owner
Clicking on any of the datasets listed as registered by you (page 10) displays the connection information (top section) and descriptive metadata (bottom section) recorded in the clearinghouse:
Editing connection information
Click on Edit Catalogue/Connection Information to edit the information needed to connect to a dataset - web address, server details etc. (this screen will be displayed automatically when first registering a dataset with the clearinghouse).
The first field, TM_Dataset_Description is used to describe the dataset in menus of available datasets, so it is particularly important that it be concise but informative.
The first three fields are obligatory, and the Notes field is highly desirable to provide some contextual information for ECAI staff and others assessing the database. You should enter either a URL (for web or ftp sites) or server details (for an SQL server database), but not both. For an SQL server database, ask the server administrator for the appropriate connection parameters.
Editing descriptive metadata
Click on Edit Existing Metadata to edit the ECAI descriptive (Dublin Core) metadata for the selected dataset. The metadata currently recorded for the dataset is displayed as follows (display may take some time - several seconds on a modem connection - as the browser must download quite a lot of program code and lists of allowable values for data entry pulldowns):
More entries here ….
If there are errors in the existing metadata, a warning message will be printed in red under the offending entry and an appropriate scheme or value (where possible) is substituted.
The metadata elements and schemes are selected from pulldown lists. When an element is changed, the list of schemes is set to those appropriate for the element chosen and the screen is redrawn. If a scheme is chosen which is a restricted list, then the list of values is available in the pulldown under Value, otherwise a textual or numeric value should be entered in the free format entry box.
When you click Commit Changes, the software checks that all required metadata elements are present, that non-repeatable elements occur only once, and that numeric values are within the prescribed range. If you get a warning message, click Back on your browser to return to the editing screen and correct the errors.
Elements can be deleted by selecting a special entry, Delete This Element from the pulldown list of elements. Elements can be added with Add Additional Metadata described on the following page.
Adding additional metadata entries
Click Add Additional Metadata to insert up to ten new metadata entries for the current dataset. The additions can be saved and the process repeated if you need to add more than ten new entries.
Set only the entries you need. Any which are left saying Select tag from list will be eliminated when the data is saved.
Adding default metadata
Default metadata (the standard set of metadata elements considered normal for registering ECAI datasets) is normally inserted when a dataset is first registered with the clearinghouse.
Exceptionally, insertion of the default entries may have been skipped. In this case, the default metadata elements can be inserted by clicking Add Default Metadata located at the bottom of the metadata listing screen. Use only if default metadata entries have not been inserted.
Exporting metadata
Click Export Metadata to create HTML format metadata which can be cut and pasted from your browser into a web editor.
The most likely reason for exporting metadata to HTML is to attach metadata to your web pages. Metadata in your web pages will promote them in web search engine listings. HTML metadata should be placed immediately after the <head> tag in your web page source (it is not visible in a browser view of your web page).
Note: if you already have HTML metadata in your web pages and want to import them into the metadata clearinghouse, you should use the Windows-based metadata editor (TM_MDE) described in the next section.
Windows-Based Metadata Editor (TM_MDE)
For tighter control of the editing process, we have written a Windows program called TM_MDE (TimeMap Metadata Editor) which can be installed on your PC.
TM_MDE provides a better user interface than the web-based version, with the following additional functionality (1 and 2 allow TM_MDE to be used as a completely stand-alone metadata editor for creating and updating metadata in standard web pages):
TM_MDE function summary
User registration
Allows registration of a new user with the clearinghouse and editing details such as email address and password for existing users. Metadata for registered datasets can only be modified by the user who registered the dataset;
Register a new dataset
Allows creation of metadata for a new dataset and registration of the dataset with the metadata clearinghouse. Metadata is created and edited in a local file under strict validation control, then uploaded to the clearinghouse. If the dataset is a TimeMap compatible database on an SQL server, the program will also create or update the TimeMap translation table on the server;
Metadata modification
Allows editing of a local metadata file and update of the clearinghouse and the TimeMap translation table (if applicable). The local metadata file can persist from a previous session, or may be downloaded from the clearinghouse;
Import/export HTML metadata
Allows import and export of standard HTML format, so that program can be used as a metadata editor for standard web pages, and can capture existing metadata for a dataset.
Installation
The program can be downloaded free of charge (download size approx. 700 Kbytes) from:
http://www.archaeology.usyd.edu.au/research/ecai
or obtained on diskette at a cost of US$20 (to cover copying and administrative expenses) from: Archaeological Computing Laboratory, School of Archaeology, University of Sydney NSW 2006, Australia, Fax: +61 2 9351 6392.
Run the EXE file (Start>Run or double-click on the file in Windows Explorer) and follow instructions to install the software. This also installs a default set of metadata definitions which will be updated automatically from the clearinghouse if the computer is connected to the Internet (see Startup).
Startup
When TM_MDE starts, it first checks to see if the metadata definitions stored in its internal database are more than 24 hours old. If so, and if you are connected to the Internet, it will automatically download an updated set of definitions from the clearinghouse. This should take less than a minute on a modem connection and ensures maximum compatibility with the evolving ECAI metadata standard.
User Registration
Before you can register datasets or update metadata in the clearinghouse, you must register with the clearinghouse as a user. You will only have access to modify connection information and metadata for datasets which you have registered using your login name and password.
New User leads to the User Registration screen shown below. You should register with the clearinghouse only once. Please avoid creating multiple identities as, at present, we have no convenient way of combining them or changing the ownership of individual datasets.
It is essential to record your password and login name for future reference, as we currently have no convenient means of finding them out if you lose them.
Neither user name nor password is case sensitive. For security reasons, passwords cannot be overly simple - for example, kdty is too short, kdty482 is too simple (a ‘word’ followed by a number), whereas boo47klet is acceptable (digits embedded in a word, or vice versa).
If you are not sure of your login name, you can find it out with Find Details. This allows searching for known parts of your login name or email address. It will not, of course, tell you your password.
Edit Details allows you to change not only your contact information but also your login name and password. Datasets registered under your previous login name will still be accessible under your new login name.
Identifying the user
When you first attempt to register a dataset with the clearinghouse or ask to edit excisting metadata, the program will check your identity through your login name and password (neither is case-sensitive):
Once you have entered your login name and password they will be remembered for the rest of the session, so there will be no need to reenter them.
Registering datasets with the clearinghouse
Registering a dataset with the ECAI metadata clearinghouse is a two-step process:
New Metadata File brings up a New File dialogue and inserts the default set of ECAI descriptive metadata entries, defined in the clearinghouse, into the selected file:
The program then opens the metadata editing dialogue shown under Editing the file (page 23).
Registering the dataset
Once the metadata has been fully edited, click Register dataset with clearinghouse. Registration assigns the file to a particular dataset entry in the clearinghouse and uploads the metadata to the clearinghouse database. Registration can only be carried out once for each new file - once registered, the file ‘knows’ which dataset it belongs to.
The program will check that you are connected to the Internet and warn if it is unable to establish a connection to the clearinghouse. It will also check that the dataset has not previously been registered with the clearinghouse and refuse to register an already-registered file/dataset (use Update Clearinghouse for a previously registered file/dataset).
As the metadata is stored in a file on your hard disk or network, there is no need to register the dataset immediately if you are not connected to the Internet. In fact, TM_MDE can be used as a metadata editor to prepare and edit metadata for inclusion in web pages without ever being connected to the Internet.
Connection details
If the connection to the clearinghouse is successfully established you will be asked to enter your clearinghouse login name and password (if not previously entered during the session) followed by the connection information for the dataset:
Select the type of dataset to be registered from the radio buttons at top left. If the dataset is a remote SQL database or TimeMap dataset on an SQL server, you will be asked for full connection data as in the example above. For other datasets (e.g. web pages or CD-ROM), a simpler form is displayed.
Click Register Dataset to complete the registration process. The program will update the clearinghouse and, if you are registering a TimeMap- compatible dataset, also upload the TimeMap translation metadata to the remote SQL server holding the dataset. You should receive a confirmation message - if you do not, the network connection may have been lost before registration was completed and you will have to repeat the registration step.
TimeMap datasets
If the dataset being registered is identified as a TimeMap dataset, TM_MDE will use the information entered on the screen above to connect to the SQL server database and create a TimeMap translation table for the dataset on the server.
TimeMap datasets must be stored on an SQL server (Oracle, Informix, Sybase, Interbase, MSQL, DB2 or ODBC compatible) and must conform to certain formatting standards. To create the TimeMap translation table, TimeMap translation metadata must be entered by clicking Insert TimeMap Translation Metadata (page 23) and entering appropriate values (see TimeMap web site for details).
Editing metadata
The program always operates on a metadata file stored as a Paradox table on your hard disk or network. This file can be one that already exists on your hard disk or network, or may be downloaded from the metadata clearinghouse.
Edit Local Metadata File brings up an Open File dialogue allowing selection of an existing file. This file can have been registered with the clearinghouse, or may simply be a local file. If selecting a registered file, be careful that you have the latest version of the metadata in the file, otherwise current metadata in the clearinghouse could be lost when the clearinghouse is updated from the file. It may be best to always use Edit Registered Metadata File and to ensure that the clearinghouse is always updated from the edited file at the end of a session. Editing of local files is provided primarily to allow editing when not connected to the Internet.
Edit Registered Metadata File first checks your login name and password (if not previously entered during the session), then displays a list of the datasets you have registered with the clearinghouse (a warning message is given if the program is unable to connect to the clearinghouse e.g. because you are not connected to the Internet):
When a dataset is selected and Download and Edit Metadata Table is clicked, you are asked to select/create a file on your hard disk or network. The file selected can be an existing file (which is overwritten) or a new file. The metadata for the selected dataset is read from the clearinghouse and stored in the file. The file is then opened for editing (next page).
Editing the metadata file
The metadata in the selected file is displayed in an editable grid:
Note Not yet Registered with Clearinghouse shown at top right for new files. If the file/dataset has been registered with the clearinghouse, the name and ECAI registration number of the dataset will be displayed here.
Neither elements nor schemes can be edited on this screen. To change an element or scheme, delete and re-enter it using Delete Selected Record followed by Add Record.
Where the scheme defined for a particular element is a controlled list, the value is selected from a pulldown. For modern dates, values are selected from a calendar. Other types of value are simply typed into the field. Numeric values are checked to see if they are within the defined range for the scheme chosen.
If you are creating a new metadata file and the dataset you are about to register is a TimeMap dataset, you should click Insert TimeMap Translation Metadata to insert the extra metadata required by the TMView program. Do not click this button for non-TimeMap datasets. See the TimeMap web site for detailed documentation of TimeMap translation metadata and the methods used to TimeMap enable an SQL server database.
Adding metadata elements
Add Record opens the following dialogue:
Selecting the Metadata Element Tag field displays a pulldown of available metadata elements:
Note that bold entries are elements required by ECAI, while grey entries are those that have already been entered. The icon on the left of the entry shows which elements can be repeated and which can only exist once in the metadata for a particular dataset.
Once an element has been selected, the Scheme pulldown will show only the schemes valid for that element. The Metadata Element Value field is sensitive to the type of scheme selected and behaves in the same way as the value field in the editable grid (page 23).
Click OK to insert the element, scheme and value into the metadata file. Click Cancel to abandon the new entry.
The ECAI metadata clearinghouse defines certain metadata elements as required. In addition, the default metadata inserted automatically from the clearinghouse definitions is identified by ?? at the start of each value. TM_MDE checks that all required elements are present and that all default values have been edited (by removal of the ?? at least) before you can exit the editing screen, and reports as follows if the conditions are not satisfied:
You can also view this list by clicking Show Missing Metadata.
Update Clearinghouse
Once editing of the local metadata file has been completed, click Update Clearinghouse to upload the revised metadata to the clearinghouse and, if applicable, to update the TimeMap translation table attached to the dataset.
A warning message is given if the program is unable to connect to the clearinghouse when Update Clearinghouse is clicked or if the file/dataset has not yet been registered with the clearinghouse. A confirmation message is given if the update is successful.
Update of the clearinghouse need not be carried out immediately - the data is still retained in the local file and can be edited repeatedly before the clearinghouse is updated.
Import Metadata
Import Metadata brings up a File Open dialogue to select an HTML file from which metadata is to be imported. The file is parsed for metadata tags ( <meta name = … ) and any tags which have valid element names, schemes and values are inserted into the current metadata file and displayed in the upper box. Invalid entries (according to the definitions stored in the clearinghouse) are listed in the lower box, together with a message explaining the error:
TM_MDE will import metadata from any standard HTML file (web page) which uses Dublin Core elements and schemes defined in the metadata clearinghouse. It will also import HTML metadata exported by the Export Metadata function described on the next page.
Export Metadata
Export Metadata brings up the following window (in this case showing the default ECAI metadata inserted for a new dataset):
The box is automatically filled with the metadata for the current file, formatted as HTML. Click on Browse to select another metadata file from your hard disk or network.
The HTML text can be saved as a file with Save Results or copied to the clipboard with Copy to Clipboard for immediate pasting to an HTML (web page) editor. Metadata exported from one dataset with Save Results can be imported for another dataset using Import Metadata.
Download metadata definitions
Whenever TM_MDE is run, it automatically tries to update the locally-stored metadata definitions from the clearinghouse, if they are more than 24 hours old.
It may be that the program is run with the computer disconnected from the Internet, in which case a stored copy of the definitions is used. If a connection to the Internet is later established, a fresh copy of the definitions can be downloaded from the clearinghouse by clicking Download Metadata Definitions.
References
For an explanation of the structuring of TimeMap datasets, and software for setting up and accessing TimeMap datasets, please refer to documentation available at the TimeMap project web site:
 |
 |
 |
School of Archaeology Home | Top of this page | |
School of Archaeology,
University of Sydney NSW 2006, Australia. +61-2-9351-2364/6394/fax 6392
Web site maintained on the Archaeological
Computing Laboratory server
+61-2-9351-3142
Content of web pages developed by individuals in their home directories is
outside
the control of the School of Archaeology and no responsibility is taken for
this material.
Copyright © 1997, 1998 School of Archaeology, University of
Sydney, except for individual user pages, or where indicated.
Developed by Ian Johnson assisted by Elizabeth
Black & Fiona
Lim