IDnature guides
New users
- Instructions are in red. Read them. Read below too.
- First time users should read How to begin and then How to identify things.
- All blue words are links.
- Most maps and images zoom at least once -- click on them.
- Black & yellow buttons process forms or move you around.
- Also use scroll bars to move.
- Use Restart in the banner to find something else in this guide.
- Use Guides in the banner to list guides.
- Use Report to submit your findings.
How to begin
- Use Internet Explorer 4, Mozilla 1.0.2, or Netscape 6, or above.
Earlier versions will work but may fail to zoom in on maps and images correctly.
Click here to upgrade to Mozilla or
Firefox.
- Make your browser window as large as possible.
- Click on "Images off" (on top banner) if things are too slow.
- All blue words are links. Click on them to use them.
- Click on images and maps to see them at a higher resolution.
- General layout:
- Top banner -- has 2-3 main links and the controls: Help, Credits, Guides, ... Search. Click on these to use them.
- Right -- presents guide, higher-resolution images, maps, or more information
- Left -- displays instructions, results, mid-resolution images, or menu
- How to use forms:
- To move between characters within a guide, use any of the following:
- Click on an
up or
down arrow
to move to the next character in that direction.
- Click on any
button and
then on a character's blue link from the alphabetical list of characters.
- Use the scroll bar on right, keeping mouse button down over scroll bar while moving it up or down.
Use the scroll bar's arrows to avoid over-shooting.
- Use the scroll bar's up or down arrows. Click on these arrows to move a little way.
Hold mouse button down over them to move faster.
- Click on words in the blue navigation bars of Discover Life and IDnature guides.
For example, click in the blue Top below.
- To move around maps, use the yellow and black arrows along the map's edges.
How to identify things
- Questions are asked on the right. Do the following:
- Select a guide by clicking on its blue link in the list under "IDnature guides" on the right.
Click on "Guides" in the banner to show this list if it is not there.
- Click on any
button to see what kinds match your choice. A list will appear on the left.
- Click on
that appears with your results on the left.
- Repeat steps 2-4 until you have completed the identification.
In some cases the guides do not yet identify things to a single kind.
In which case, several names will appear on the the left and
you will not be presented with any new question when you click on "simplify."
- Click on boxes to select characters in any order.
If in doubt about an answer, either skip the question or click all boxes that you think may be possible.
While it is best to pick a single state for each character presented, 20q is forgiving.
If you're unsure, then
- skip answering and go to next the character;
- choose several possible states;
- click the "not" box and one or more state boxes to exclude ones that you're sure don't apply.
Banner controls
- Help -- shows this page.
- Credits -- credits & information about guide.
- Guides -- shows the list of guides that you can select.
- Restart (appears only after you select a guide) -- restarts guide for next identification
- Checklist (appears only after you select a guide) -- presents a list of kinds within the group.
Click on any blue name in a checklist to link to images and more information.
- Images off/on
- Use "Images off" or "Images on" in the banner to switch the guide's images off or on.
Switch them off if you have a slow Internet connection, an old, slow computer, or a low resolution monitor.
- If images are off, you can display them by clicking on blue character names.
- Menu (appears only after you select a guide) -- puts the following advanced features on the left:
- Images & more info -- Select name(s) and then "submit" to see links and images.
- Characters -- select name(s) and then "submit" to see characters as text and icons.
- Compare differences -- select 2 or more names and then "submit" to see how they differ.
- Building tools -- only authorized guide builders can use these tools. For other users, they have no effect.
See Building guides & checklists
Report -- report your observations -- under development and testing.
Search -- text search to find links and images.
To see "Hints on how to search," leave text box that appears blank, then click "search" button.
Building guides & checklists
Building guides and checklists is a two step process that involves
(1) putting information into eXtended markup language (XML) files and then
(2) making database (db) files from the XML files. The XML files are text based and can be read and easily manipulated
by humans, either with a text editor or with a set of Web-based building tools.
The db files are used by the 20q software to run the guides
quickly. They can't be read by text editors and are built with the "refresh" tool (see "Building tools" below).
Security
To use the building tools, first select a guide and then click on "Menu" in the top banner. The building tools are
displayed on the left. Everyone can use these tools to manipulate the XML files on the outside of a firewall.
However, only authorized users can make permanent changes to the XML files through a process
that involves email transactions or authorized IP addresses.
If email is used, the tools store changes on the outside of a firewall and send an email with a PIN number to a specified email account.
Permanent changes occur when the email with the PIN number is returned to the server and is authenticated against two files,
one containing authorized email and IP addresses (20q_ALLOW) and the other, the transaction PIN numbers (20q_MAIL_PIN). The system ignores
emails that are from accounts that are not authorized to change a specific guide or that contain a PIN number that is incorrect.
Guide builders who reply to an email must include the original text with the PIN number in their reply or it will be ignored.
Making changes is easier if the guide builder is working at authorized IP addresses. They simply use the tools and then "refresh"
to make the changes permanent.
Guides versus checklists
A guide and its associated checklist share a single XML file and are built with the same tools.
However, they differ in which kinds of organisms they display.
We define a kind as a unique combination of an organism's name, morph, and sex.
To appear in a guide a kind must have at least one character-state. Kinds without any character-state attribute appear only in the checklist.
Guides distinguish between morphs and sexes. Checklists do not, listing only kinds with different names.
The order in which kinds are displayed in checklists has several options, is in flux, and is currently not documented.
Generally, kinds are listed alphabetically by scientific name; however, in some cases,
kinds may be listed in groups based on their "path," a variable that is set with the "score" tool (see "Building tools" below),
that generally reflect groups such as orders and families.
If you are an expert in a group and wish to build a guide or help with one,
please contact John Pickering.
Process -- one of elimination
Guide users and builders may have to switch their thinking to work with IDnature guides.
Most other identification keys use a process of selection. In contrast, IDnature guides work by elimination.
This difference gives IDnature guides a fundamental advantage in the practical world where some questions can't be answered.
Antennae banded? Oops. They've been knocked off.
How to deal with unknown or unscored characters?
-
Rule for users -- don't guess
Most conventional keys are linear and are dichotomous. Users are forced to answer questions in order of presention,
typically selecting between two options. Failure to answer a question means failure to proceed or requires working the key in both directions,
hoping for later information that will help decide which answer to pick. Fortunately, interactive keys such as IDnature guides
are non-linear. Users can answer questions in any order. Failure to answer a question means only that the power provided by a particular answer
is lost. Users can skip any question and proceed with the next. In fact, users should skip all questions unless they are sure of the
answer. A wrong answer can eliminate the kind being identified from the guide. No answer leaves it in.
Unscored characters
In the ideal world, guide builders should score every kind for at least one state of each character.
However, 20q is forgiving and doesn't require that all kinds have a state for all characters.
It will not eliminate a kind if it doesn't have a state for the character under consideration. In short, unscored characters
make the identification process less efficient. They do not affect the identification itself.
In cases where a character does not apply to a kind, we recommend that guide builders score "None",
rather than leaving the character unscored. For example, we recommend a single character "Leaf latex color"
with "None" and Red, White, Yellow, Clear, etc. This is more powerful
than two characters "Leaf latex present" with states of "Yes" and "No" and a character "Leaf latex color" without
a "None" character.
Rule for builders -- anticipate user errors
Builders should score all states that apply to a kind and should anticipate
user errors and score states so that they accept some variance and error in answers. For example,
for a plant that typically flowers in April, the guide builder may want to score
March and May too, thus bracketing for latitudinal, elevational, and between-year climatic influences.
Don't allow your users to exclude something that they shouldn't.
If a character is difficult, then the above rule can sometime lead to guide builders wanting to score
a kind for all states. From the users view, this is equivalent to leaving the kind unscored.
Building tools
Before reading this next section, please familiarize yourself with
20q XML Tags & File Structure.
The following tools appear on the left under "Menu."
To use a specific tool, click on the box to its left and then on the "submit" button above it.
Follow the instructions that appear in red after you select a tool.
- images & more info -- lets you compare images and other information of the kinds.
Select one or more kinds by clicking on the box next to their name, click on the box
to the left of "images & more info," then click on "submit" button. Images will
appear in the lower right frame.
- resolve -- enables you to see how completely the guide can resolve individual kinds.
It has the following two modes:
- General
-- allows you to evaluate all the species. Simply click on the box
to the left of "resolve" and then click on the "submit" button. This will show all the
kinds that a user can't resolve to a single kind. After results appear in the lower right frame,
guide builders can transfer the results for each species to the Menu for further processing
by clicking on the "left arrow" buttons that appear to the left of the names. With this mode,
click on the box to the left of "verbose" to get more information about the kinds that don't
resolve.
- Specific
-- allows you to evaluate a single kind. To do this, click on the box to the left
of a name, then click on the box to the left of "resolve," and finally on the "submit" button.
All the kind's characters will appear in the lower right frame. Once these appear,
you may unselect specific character-states by clicking on their boxes, if you wish.
Then click on any "identify" button to see whether the kind resolves or doesn't.
characters -- allows you to view all the character-states scored for one or more kinds.
differences -- allows you to compare how two or more kinds differ in their character states.
refresh -- refreshes both the guide & checklist's database from the XML file.
Note: Earlier versions required builders to refresh_guide & refresh_checklist. This tools combines both features.
undo -- allows the builder to go back to a previous XML version. Use this tool if you make a mistake or if a
guide doesn't work after using "refresh."
Most recent versions are listed at the top by date and GMT time (yyyymmddhhmnss) and file size.
After using "undo" to save a previous version, reply to the email, and then "refresh."
has -- this is a powerful tool that allows guide builders
to identify kinds scored for a specific character or set of characters.
Unlike the logic behind the guides presented to users with the Banner's "Restart"
option, which does not exclude kinds that haven't been scored,
this tool ignores kinds that haven't been scored for selected characters.
Thus, you can use 'has' to figure out what has and has not been scored for particular character-states.
For each row of character-states, this tool presents the following three options:
- only -- determines EXACT MATCHES, that is to say,
kinds that match must be scored for all specified states within each character and no more.
Hence, this option allows you to examine a combination of multiple states with each character simultaneously.
Use it, for example, to determine which kinds are scored as monomorphic for a state, or alternatively,
which have an exact set of polymorphic states.
- has -- determines ANY MATCH, EXCLUDING UNSCORED KINDS. It displays kinds that have
at least one of the clicked states, independent of whether they have more than one.
- not -- reverses the logic of 'only' and 'has.' As with the guides presented by 'Restart,'
you may use 'not' by itself to negate state boxes. However, when you use 'not' in combination with 'only' or 'has,'
the kinds returned won't include ones that do not have at least one state scored for each selected character.
Here are a few (tricky) examples to help you use this tool:
- To determine what kinds haven't been scored for a character, use 'not' by itself, without clicking any
state box. This is a shortcut. It yields that same result as if you clicked 'not' and all the state boxes.
- To determine what kinds have been scored, use 'has' by itself, without clicking any state box.
This is a shortcut. It yields that same result as if you clicked 'has' and all the state boxes.
- It gets contorted!!! Clicking 'has' and 'not' by themselves, without clicking any state boxes, yields no matches.
However, 'has' and 'not' in combination with at least one state box will yield all species that have been
scored for the character, but don't have the selected states, unless they are polymorphic for non-selected states.
In short, Sam, you're right. When all else is said and done, rather than reading the manual, we simply recommend
that you play with it, test your logic, and have fun.
score -- use to add kinds and change information about them.
The form that appears has three sections:
- Index (at top in blue).
- Header -- for entering names, authority, level, morph, sex, links to other sites, & notes.
- <link>
In the header section of "score," 20q presents guide builders with a number of boxes to link images, maps, and other information
to a kind. This is done with <link> tags
(see details).
These links are combined across morphs and sexes with the same <name> value.
Users see them in the order that they are stored in the XML file,
or files, if the first file uses <include> tags.
- Characters -- for scoring character-states.
- Character order -- 20q presents "unscored" characters first, then previously "scored" ones,
and finally "New" ones. 20q first presents unscored and scored characters
in the order that they are specified in "initial characters"
(see "controls" tool) and then alphabetically for all characters that are not initial ones.
The index at the top of the "score" page lists both unscored and selected ones alphabetically.
- New states -- under each character are 4 text boxes to add new states.
If you wish to add more than 4 new states to a character,
enter them as different "New" characters and but entering the same character name.
If you wish to score just a subset of characters, you can group them together by setting them to be "initial characters" with
the "controls" tool. Also see Character-state naming conventions and
Templates below.
drop -- removes a single kind (name-morph-sex combination) from the XML file.
This tool reqires that you click on a box to the left of the list of names that appear below the tools.
If there is more than on kind associated with a name, you are present a choice of which to remove.
If you wish to remove both a male and a female, for example, you will need to use this tool twice.
rename -- rename a single kind.
This tool allows you to rename a kind to another name+morph+sex combination. Follow the instructions that appear in red.
Within the XML file, it replaces the old kind with a new one, maintaining the kind's order within the file.
It does not chance the authority or other tags.
score_if (score_all) -- is a powerful tool that conditionally scores the kinds in the XML file for an attribute (character-state)
based on a set of rules that the guide builder sets. The rules are based on each kind's scientific name, path, morph, sex, and
existing character-state attributes. On the left, the builder specifies the character-state to be scored throughout the guide.
On the right, the builder specifies the rules. There are five rules, all of which must be true before the character-state
is added to a kind's attributes. Each rule has three options:
- any makes the rule true.
- if= uses the information supplied under it and is true if the rule then applies to the kind.
- unless= is the negative of "if=," that is to say, it is a "not if=."
In the "scientific name" box, a guide builder may put multiple names, selecting them with the check boxes
in the list that appears below the Building Tools under "Menu." Separate multiple names with commas.
If a species name, genus name, or group name, such as familiy, ends with an underscore, then the rule is applied not only
to the name but to all kinds that are members of in the named group. Names are checked against the
<name> and <path> tags.
Ending a specific name with an underscore applies the rule to the species and all its subspecies.
Examples:
- "if=Cinnamomum" would be true only for the genus Cinnamomum per se, while
- "if=Cinnamomum_" would be true for Cinnamomum and for all species, subspecies and morphs within the genus
because of the underscore.
- "if=Brassicaceae_" applies to all kinds in the family Brassicaceae.
- "unless=Brassicaceae_" applies to all kinds NOT in the family Brassicaceae.
- "if=Brassica_, Eruca_" applies to all members of these two genera, which must be separated by a comma between their names.
HINT -- Use "score_if" rather than "score" to score a character once a guide is populated with kinds.
Steps:
- score one or more kinds that you specify in the Menu for a state of a character,
- keep repeating Step-1 for other states of the character,
- refresh, and then
- score all remaining kinds for an alternate state by setting the "character or state" rule to "unless=Your specific character."
Unless a kind has already a state for the character, Step-4 will assign it the alternate state. For example, imagine you have a species
with green eyes, one with blue eyes, and the rest with black eyes. Use score_if to score character "Eye color" as state "Green" for
"Species 1." Then score "Eye color" as "Blue" for "Species 2." Then Refresh. Finally, score_if "Eye color" as "Black"
with the "character or state" rule "unless=Eye color". All kinds but the first two will be scored "Black."
To score all kinds in a guide, (1) select score_if, (2) enter the character-state on the left,
(3) leave the four rules on the right at their default "any" setting, (4) click the "save" button, and
(5) once the changes are on the server, refresh.
drop_if (remove_state) -- removes a state or character for the kinds in the XML file for which the specified rules apply.
See score_if for how to specify rules.
Note: drop_if only removes a character from the kinds' attribute and not from the character in "c_initial".
Use the "controls" tool to remove c_initial characters.
rename_if (rename_state) -- renames a state or a character for the kinds for which the specified rules apply.
See score_if for how to specify rules.
Note: Doesn't rename a character in "c_initial". Use the "controls" tool to manage c_initial characters.
image_tools -- allows builders to manipulate images and then link them to character-states in a guide or put them on a kind's page.
Features include: crop, rotate, mirror, sharpen, contrast, bright, saturate, hue, and draw.
The "draw" feature allows you to annotate character-state images with a circle or line of specified thickness and color.
You may specify a circle's size and a line's edge of origin with the options.
edit_explain -- allows builders to add and remove a guide's character-state image.
By default "edit_explain" links character-state images to _ALL_ rather than to a specific kind in the guide.
However, while not recommended, you may link a images to a kind by selecting one from the menu.
To remove an image, switch it "off" and then save.
edit_text -- allows builders to add and manipulate text on a kind's page.
Steps to adding text to the guide
From the Discover Life homepage:
1. Click on IDnature Guides.
2. Click on Guide you wish to make changes to.
3. Click on Menu at the top of the page.
4. Click on the first letter of the name of the genus you would like to work on within the menu frame, then click submit.
5. A list of genera will appear in alphabetical order by genus and species name.
6. Check the box next to the genus and species you would like to work on.
7. Scroll to the top, click edit text, then submit.
If you scroll down you will see fields such as overview, kinds,
identification, etc. When you are finished making your changes, click
save on the left side of the main frame. A screen will appear that says
"Thank you. Made changes on server." After this screen appears, hit
refresh on the left side of the screen in the menu area. This will take a
minute or two, as it has to refresh the entire guide.
If your IP address has not been authorized by Discover Life's system,
then the screen should say "Thank you.
Changes have been sent to [email address]." If this is the
message that appears, then the person who receives the email must reply
to the email with the original text included before the changes are made
on the server.
controls -- to edit the guides <20q_controls> tags:
<who_default>
<email_default>
<base_path>
<levels_default>
<c_initial> -- initial characters
<include> -- Note: the "controls" tool doesn't yet allow guide builder to add and remove <include> tags.
TO ACTIVATE CHANGES -- The changes that you make with the "control" tool may not take effect until after you reply to the email,
use the "refresh" tool, and then start the guide again from scratch. Start the guide again by selecting "guides" in the banner
and then clicking on the guide. This is an exception. All other tools take effect after you click on "refresh" and then "Menu."
credits -- To edit the credits that appear on the top of a checklist or when "Credits" is selected on the banner,
edit the HTML code that is within <checklist_credit> and <guide_credit> tags. See
HTML to get started with HTML tags.
get_ip -- Displays the IP address that user's browser is exporting to Discover Life's servers.
This is not necessarily the IP address in the user's computer, which may be an intranet address.
Instead it may be an external one exported by a firewall.
import -- allows guide builders to import data into IDnature Guides's XML format.
Information can be imported from Lucid, DELTA, and text files exported from
various spreadsheet and database programs. This tool can only be run from authorized IP addresses.
If you wish to import data into an IDnature guide, please contact
John Pickering, who can help you through the process.
- from text files
To import information into 20q from a text file, the text file needs a header line followed by a line
for each kind that you wish to import. Your text file may be placed anywhere on the Web or can be emailed to Discover Life.
Its name should not contain spaces or special characters.
The header line should name the fields in the order that they appear in the rest of the file.
Fields may be separated by dollar signs, tabs, or commas,
but commas are not recommended, as they often are used within fields.
Lines should be separated by new line characters. Carriage returns are ignored. As an example:
path$name$authority$level$common_name$links$notes
is a header that would import these seven fields. The first two are required; the others optional.
If 'level' is not specified, it defaults to 'species'. Use semicolons to separate multiple common names and synonyms in
the 'common_name' field.
You can import character-state information with header fields starting with CS (in capitals), followed by a colon,
and then the character's name. In the corresponding fields in subsequent lines, put the state. Thus,
path$name$authority$common_name$CS:Wing number$CS:Leg number
is a header line that allows you to import two characters, "Wing number" and "Leg number."
Use a semicolon within the "common_name" field to separate multiple common names and synonyms.
Similarly, use semicolons to separate multiple polymorphic states within a CS:character field.
- advanced options
There are options to drop subspecies ($flag_drop_subspecies=1) and to drop multiple species ($flag_multidrop=1)
from a text list specified my $multidrop in the txt2xml subroutine.
Output from the import tool is put in file="$guide.xml3", which, if correct, should be renamed to the guide's xml file ("$guide.xml").
export_xml -- &act=xml3 -- checks existing XML file; flags any errors in red; builds xml3 file for export.
This tool is useful to check XML that has been created with a text editor. It does not appear under menu and
must be placed in the URL. For example, "http://www.discoverlife.org/20/q?guide=Whatever&act=xml3"
will process the XML file for a guide named "Whatever".
attachment -- send the XML code that is created as an attachment to the email that is sent.
verbose -- displays the XML being processed by "refresh" tool. This may be useful in error checking,
but is slow with large guides.
Character & state conventions for choosing, naming, and presenting characters
Choosing, naming, and presenting the appropriate characters and their states is the hardest part of building a guide.
Our goal is to use characters and states that the average 4th grade student can understand and recognize.
While in some cases it may be impossible to avoid technical terms, please try. Also please try to
avoid characters that require special equipment and methods that may not be generally available outside
of research instiututions. For the most part we hope that binoculars, a hand lense, or a school microscope
will be sufficent to successfully use the guides.
In choosing characters and states, there are several considerations that stem from the advantages and limitations
of computer based identification. These are
- Character order
Conventional keys are typically dichotomous and present sets of two characters in a linear sequence.
In contrast, the order in which 20q presents characters depends on which states a user has already selected.
The software presents a special category of characters, initial characters, when the
guide is started. Initial characters appear in the order specified by the guide builder with the
"controls" tool (see above) and are listed in order of appearance in the XML file between tags.
After the "simplify" button is pressed, 20q only presents the characters and states that have power to
distinguish kinds from each other. These remaining characters are presented in alphabetical order.
Other characters and states are not presented. While there is no limit to the number of initial characters,
we recommend that a guide builder choose 5 to 10 initial characters, so that the guide is presented to the user
quickly.
- State order
20q uses the following rules to order the states that it presents:
- States that start with a number appear before ones that start with a letter.
- States that start with a number are sorted numerically.
- States that start with a letter are sorted alphabetically.
- States with "mm" are before those with "cm", which are before those with "m".
- Special case -- a state that starts with "Under", a space, and then a number (note it's Under not under)
-- appears before states sorted by numbers.
- Special case -- a state that starts with "Over", a space, and then a number (note it's Over not over) --
appears before ones starting with letters and after ones starting with numbers.
- Special case -- a state that starts with the word "Other", (note it's Other not other) --
appears after all other states.
Thus, "Under 1, 1, 2 - 99, 100, Over 100, None" should work. As does "1 cm - 1 m, 1 m - 10m".
If units such as cm are the same in all states, put the unit once at the end of the character name rather than in all
of the state names. If state names need changing to take advantage of these rules, use the rename_if tool.
- Indexing characters
The index always lists characters alphabetically, even for the initial characters that are presented in
the order specified by the guide builder. To keep associated characters together and so that they are
easily looked up in the index, it is important to name them so that they index well. For example,
if you name a character "Wing number" rather than "Number of wings," it will group with other
"Wing" characters and not with other "Number" characters.
- Number of states
A major advantage on interactive keys is that they can have multiple states and are not dichotonous.
So the question is how many states to have per character. For practical reasons,
if images are going to illustrate a state, then we recommend that
a guide build aim to have 4 or 5 states per character. While there is no restriction on the number,
if there are too many, then the guides become more difficult to use. The human mind
is limited in the number of patterns it can recognize simultaneously. Too many states also require
the user to scroll from left to right, rather than seeing all states together.
- Consolidating states
To help guide builders consolidate states if there are too many, 20q provides a summary of how many kinds
have each state. This summary appears at the bottom on the left after clicking on "identify" before selecting any state.
To maximize the power of each character, try to consolidate states so that each remaining state contains an
equal proportion of the kinds. Thus, it is generally better to have 4 states that each describe 25 kinds,
than 1 state that describes 97 and 3 others describing a single kind.
- Length of text of characters and states
Basic rule -- keep 'em short!!! Because there is limited space on monitor,
you should put as much of the character-state combination in the character name
and as little as possible in the state name.
Thus, "Thorns present"--"Yes"/"No" not "Thorns"--"Present"/"Absent."
And, for example, "Over 5" not "More than 5."
- Symbols -- good vs. bad
Because of the conventions computers use to communicate, 20q is limited in the set of symbols that
it allows in character and state names. It allows all letters ( a-z A-Z ), numbers ( 0-9 ), underscores ( _ ), hyphens ( - ), some punctuation,
such as spaces ( ), commas ( , ) and colons ( : ), and a few other symbols, including the divide or foreslash symbol ( / ).
We recommend that you only use these symbols.
20q can fail if characters and states contain
apostrophes, plus signs, equal signs, brackets, parentheses, ampersands, and other symbols.
When in doubt, spell it out. Thus, use
"Over 3 mm" not "> 3 mm". Similarly, use "Red and black" not "Red & black".
The newest version of 20q converts ampersand ( & ) to "and," less than ( < ) to "under," greater than ( > ) to "over,"
and percent sign ( % ) to "percent."
It blocks plus sign ( + ), semicolon ( ; ), grave accent ( ` ),
apostrophe ( ' ), quotation mark ( " ), vertical bar ( | ), asterick ( * ), question mark ( ? ), tilde ( ~ ),
caret ( ^ ), left parenthesis ( ( ), right parenthesis ( ) ), left square bracket ( [ ), right square bracket ( ] ),
left curly brace ( { ), right curly brace ( } ), dollar sign ( $ ), newline, and carriage return by converting them to spaces.
The equal sign ( = ) is a no-no; spell it out as "equals."
- Capital versus lower case letters -- start characters and states with capital letters
20q distinuguishes between the case of letters. "Red and black" is a different state from "Red and Black".
As a convention, so that guides can exchange character-states and index correctly, we request that
guide builders start characters and states with a capital letter and do not use capital letters
elsewhere unless it is a name or required for emphasis.
The newest 20q versions automatically capitalize the first letter of new characters and states.
Note: Currently states are ordered by a sort routine that needs to be improved. The months, for example,
are presented alphabetically, rather than chronologically. In due course, we will improve the order
that characters are presented based on categories such as numerical, chronological, etc.
- Character name format: "Part" + "View" + "Modifier"
We recommend the following:
- Part: Use everyday terms rather than specialized ones whenever possible.
For example, for insects, use "Foot" not "Tarsus."
Use short, singular terms that alphabetize by major
body part, such as "Leg number" not "Number of legs."
- View: Use "front, rear, top, bottom, side," assuming the organism is upright.
By convention, we use "Back" as a body part, not the opposite of front.
Thus, on a bee or butterfly, use "Wing front top pattern" or possibly
"Forewing top pattern" for the upper surface the wing closest to the head.
Avoid "Front wing top pattern" because it won't alphabetize with other wing characters.
-
Modifier: Try using "number, pattern, color, texture, shape, type, present."
And keep it simple -- use "Thorax top pattern" to describe the pattern on the top of an insect's thorax, not "Mesoscutum dorsal pattern,"
as our earlier guides do!!!
- Adding text
Put text describing a state in a file with the same name as the
image file, except that instead of using a suffix such as ".mx.jpg", use ".txt".
Eventually we plan to put a tool under the Menu that will allow guide builders to add this text
via the Web. To see how txt files work, click on one of the thumbnails on
Insect orders,
a guide developed by Lucid. The text appears on the left, below the 240 pixel resolution image.
Templates
To help you build guides, share character-states with other guide builders, and score kinds rapidly,
IDnature guides use the following:
-
_ALL_
is a "ghost" kind. 20q does not present it to users in the guides or checklists. Instead it is
used by the guide builder. If present, it appears in green as "ALL states & images" immediately
below the Menu's Building Tools. It is a means of storing characters, states, and images that may or may not
be in the attributes of the guide's other kinds. It is particularly useful when first setting up a guide and allows
guide builders to assemble characters and states from other guides using <include> tags
(see below). We recommend that guide builders put all state <image> tags within _ALL_'s
<attributes> tags, rather than scattering them throughout the XML file within the
<attributes> tags of the guide's other kinds.
Use the "score" tool to create an _ALL_ ghost. Simply enter "_ALL_" in "scientific name" and save.
Use "drop" to remove it.
-
<include> tags
allow guide builders to include other XML files. In addition to allowing a
guide builder to create a large guide/checklist by including kinds form multiple small ones,
these tags allow guide builders to share libraries of characters and states.
To do this, a guide builder should do the following:
- include other guides
- score for _ALL_ (see above), selecting the character-states from the other guides that are
useful
- then remove the <include> tags.
Warning: <include> tags may cause some confusion if a kind's XML is store
in a guide and also in any of its included guides. Two or more
copies of a kind will appear when "score" or "drop" are used if any of the following differ among XML files:
(1) authority, (2) easy name, (3) level, (4) common names, and (5) path. To have only one version of a kind appears, make
sure that these five variables are exactly the same across XML files. Or better yet, just put the kind in a single XML file.
Note: Guide buiders still need to use Unix to add and delete <include> tags.
Eventually they will be able to edit them from their browser with the "controls" tool under "Menu."
-
_TEMPLATE_ is a ghost like _ALL_. It is not visible to guide or checklist users,
only appearing in green under the Menu's "Building Tools." A _TEMPLATE_ ghost is
any kind with a "scientific name" that starts
with "_TEMPLATE_", such as "_TEMPLATE_Bombus" or "_TEMPLATE_Brassicaceae". They are created with the "score"
tool and removed with "drop". They allow a guide builder to define a set of characters and states. Once a
_TEMPLATE_ is created, call it up with "score," change the "scientific name" from "_TEMPLATE_whatever" to
the name of a new kind, score the new kind, and then save. The new kind's XML contains whatever was in the _TEMPLATE_
and the changes added while scoring.
Further information
For further details on why and how to build guides and checklists, see
- Objective --
The Polistes Foundation's plan to build IDnature guides for 1,000,000 species by the year 2012.
- First steps --
Email to guide builders in Pretoria, South Africa, October, 2002.
- Building Web Pages & IDnature guides --
for other help working with computers and the Web.
Adding images -- type, resolution, and file name
This section refers to the older JP80000 image system.
The newer EL_DP and I_ will be documented elsewhere.
20q uses <image> tags to link images on the Web into the guides.
While any image URL could theoretically be used to illustrate the guides, Discover Life and IDnature guides
function better if guide builders follow the following conventions:
- Symbols in file names
It is a good idea to use only the alphabet, numbers, underscores, colons, periods, and commas in image names,
because Image Tools does not accept other symbols. It is a bad idea to
use spaces, pluses, minuses, question marks, apostrophies, stars, parentheses, and other special characters.
Note that 20q is case sensitive. "A" and "a" differ, for example.
- Type of images
We recommend using jpg or gif images because virtually all browsers support them.
- Resolution
Our aim is to present the highest quality images possible. Thus, we recommend that
guide builders use the highest resolution that their technology permits.
To present images to users rapidly, and because of space limitation on monitors, 20q serves
5 different resolutions of each image. These resolutions are
- 80 pixels high -- thumbnails that illustrate states with the guides
- 240 pixels wide -- displayed on left if user clicks on thumbnail
- 320 pixels wide -- displayed to illustrate a kind at low resolution
- 640 pixels wide -- displayed to illustrate a kind at high resolution
- maximum -- used in zooming in to a particular part of a lower resolution image
Our image processing tools can process a directory (folder) of maximum resolution images
into the 4 lower resolutions with the a single command, "build_images all".
- Fields in file names
While guide builders may link images into the guide whatever name they wish, 20q's advanced functions
depend on image files being named with 4 fields separated by periods. These fields are
- Unique image identifier -- e. g., JP80089_34
-- is a unique IMAGE identifier that distinguishes an image from all others.
This is used by 20q to link image with associated information.
- Desciption of image -- e. g., "Equus_burchelli,_Plains_Zebra"
-- 20q automatically displays components of this description below the image when displaying the image.
This description can have sub-components separated by either commas or underscores.
Within the description, we recommend including some or all of the following components:
- Unique specimen identifier -- e. g., "UGCA123456" -- which differs from the unique image identifier
- Scientific name -- e. g., "Equus_burchelli" -- separate words with underscores NOT SPACES
- Morph & sex -- e. g., "South_African_female"
- Character:State or Part or view -- e. g., "Pattern:Striped" or "Head" or "Leaf top"
- Common name -- e. g., "Plains Zebra", or "Wilson_Warbler" but NOT "Wilson's Warbler",
or "Blackcapped Chickadee" but NOT "Black-capped Chickadee" -- don't use apostrophies or minuses!!!
Because the Desciption is indexed by Google, we recommend that you include at least the scientific name.
- Resolution -- use either 80, 240, 320, 640, or mx (for maximum, which is used by 20q's zoom tool).
- Type -- use either lower case jpg or gif (or png).
The file name must end with the Resolution and then Type fields seperated by a period. To help Google and other search engines index an image, we
recommend that you put the Description before the Unique_image_identifier field; although this is not required.
Thus, Description.Unique_image_identifier.Resolution.Type is the recommended order.
Our software automatically provides multiple links below photographs based on the Unique_image_identifier and Description.
The Zebra image above is named "Equus_burchellii,I_JP2119_1.jpg" and links to a larger image and its metadata through the URL
"http://www.discoverlife.org/20/q/?see=I_JP2119".
To help seach engines find this image, we put "Equus_burchellii" before the unique identifier in the image name.
For further instructions on putting images on Discover Life and in IDnature guides, please see
-
Illustrate character-states
-- tells you what to put in the guides XML <image> tags to link thumbnails to higher-resolution
images.
-
edit_explain tool, under "Building Guides," allows you to edit
the <image> tags for individual character states.
-
File transfer for an explanation of how to get your images
onto Discover Life or another Website (under construction).
-
Directory stucture for an explanation of where to put them within
a Website (under construction).
-
Image Tools, software described under "Building Guides" that we are writing to
automate and combine the above tasks and replace the need to purchase commercial software, such as Photoshop.
Web services -- how to put our tools & content on your website
Overview
Discover Life provides web services to share its tools and content through other sites.
Without seeking permission, any non-commercial website may customize links and forms in their pages
to use the services described below. Commercial sites should not use them unless they have permission.
Here we explain how to write HTML <a href...> tags (GET queries)
and web forms <form...> (POST queries) to pull up Discover Life content from within your site.
You can customize these queries so that
Discover Life generates navigation bars that link back to your pages. Thus,
your users can easily return to your pages after viewing Discover Life content.
The basic principle is simple. You will use HTML to pass variables to Discover Life's
servers that will then generate pages from our network of contributing databases.
Each query will call a server and tell it what to do based on the variables you send.
Example Let's consider the component parts of the following HTML:
<a href="http://www.discoverlife.org/20/q?guide=Trees" target="_top">Tree and vine guide</a>
which hyperlinks our North American tree and vine guide (Trees)
to the text Tree and vine guide.
The basic structure of this is
<a href="URL" target="_top">TEXT</a>
where
URL is http://www.discoverlife.org/20/q?guide=Trees
and
TEXT is whatever text you wish to hyperlink, in this case, Tree and vine guide.
The URL calls a server (http://www.discoverlife.org), a specific program (/20/q),
and passes a single variable (?guide=Trees).
We recommend that you use target="_top" in <a href...> tags
as shown, otherwise you will have a problem when using frames.
|
Selecting a Discover Life server
You can call a Discover Life server dynamically or select a specific one.
The link in the example above calls
http://www.discoverlife.org/20/q?
which selects a server dynamically.
Alternatively, you can call a specific server, such as
http://stri.discoverlife.org/mp/20q?
Generally we recommend the first method. It allows Discover Life to load balance across its servers.
Eventually, as our regional centers come on-line, we intend for you to be able to call servers by region and by language.
Customizing navigation bars
Set &btxt, &burl, and &bi variables to customize navigation bars.
These variables let you present information within your site's framework
so that it is not obvious to your end-users that Discover Life is working behind-the-scene.
Customized navigation bars return users to your site after they have viewed material from Discover Life.
While we credit photograhers and other content providers, you can in effect hide Discover Life's role to end-users.
We do this to encourage other sites to share our content widely.
For partner sites that contribute information to Discover Life, we use a single variable, &b,
that customizes our navigation bars and icons to match the look-and-feel of our partner's site.
Please contact John Pickering if you wish us to set up this custom feature.
The following uses &btxt and &burl to link
http://www.your.org/page.html
to Your Organization in the tree guide's banner:
|
<a href="http://www.discoverlife.org/20/q?guide=Trees&btxt=Your+Organization&burl=www.your.org/page.html" target="_top">
|
Tree and vine guide</a>
- &btxt
-
No spaces
Do not put spaces in &btxt. Use plus signs instead.
Thus, it's Your+Organization not Your Organization.
- &burl
-
http://
It is unnecessary to put http:// in &burl. It is the default.
Thus, www.your.org/page.html and
http://www.your.org/page.html are equivalent.
-
Linking to cgi scripts
To link to a page generated by a cgi script, change the ?, &, =, and + characters as follows in the &burl variable:
? to :;q:; (i. e., colon semicolon q colon semicolon)
& to :;a:; (i. e., colon semicolon a colon semicolon)
= to :;e:; (i. e., colon semicolon e colon semicolon)
+ to :;p:; (i. e., colon semicolon p colon semicolon)
For example, the HTML source code (without spaces or new line before &burl and &btxt)
<a href="http://www.discoverlife.org/20/q?search=Abelia+chinensis
&burl=
taibnet.sinica.edu.tw/chinese/taibnet_species_detail.asp:;q:;gen:;e:;Abelia:;a:;spe:;e:;chinensis
&btxt=TaiBIF
">
<img src="http://www.discoverlife.org/favicon.jpg" border=0>Abelia chinensis</a>
yields
Abelia chinensis
This uses Discover Life's search variable to generate a species page with navigation bars
that link to a page generated from the TaiGIF database at Academia Sinica, Taiwan.
After parsing the &burl value
taibnet.sinica.edu.tw/chinese/taibnet_species_detail.asp:;q:;gen:;e:;Abelia:;a:;spe:;e:;chinensis
it becomes
http://taibnet.sinica.edu.tw/chinese/taibnet_species_detail.asp?gen=Abelia&spe=chinensis
-
is Discover Life's 16 x 16 pixel icon.
Above we used <img src="http://www.discoverlife.org/favicon.jpg" border=0>
to include this image.
You may download and use this image from
http://www.discoverlife.org/favicon.jpg.
- &bi
This variable lets you put your icon in the IDnature guide banner. Thus,
<a href="http://www.discoverlife.org/20/q?guide=Trees&bi=www.your.org/your.jpg" target="_top">
substitutes the image specified by http://www.your.org/your.jpg for the banner's default butterfly.
|
-
http://
It is unnecessary to put http:// in &bi. It is the default.
Thus, www.your.org/your.jpg and
http://www.your.org/your.jpg are equivalent.
-
Icon size
You may specify an icon of any size. However, we recommend one of 69 pixels high
to match the height of the water lily on the banner's left, which is 80 pixels wide and 69 high.
|
|
Put IDnature guides on your Web pages
Non-commercial sites are welcome to put links to IDnature guides on their pages. Other sites must have permission before doing so.
Before reading this section, please familiarize yourself with the overview
and examples of how to use Discover Life's web services (see
Web services overview).
|
|
Linking to Discover Life's images and slide shows from your Web pages
Discover Life is assembling a collection of high-resolution images from our contributors.
We process and store these in our image center at five resolutions,
assigning each image a unique identifier.
Here we explain how to call individual images, sets of images, make slide shows,
view metadata associated with images, and change resolution. We assume that you are
familiar with the overview and examples of using Discover Life's web services. If not,
please goto Web services before continuing.
- Copyright and terms of use
Non-commercial sites are welcome to link to these images without seeking permission,
so long as they (1) display any copyright text and links associated with each image
and (2) respect the copyright and terms of use as specified by the contributor(s).
Commercial sites must have permission from Discover Life or an image's contributor
before using an image. Our contributors retain copyright and all legal rights to
the use of their images.
- On-site versus off-site images
Discover Life's species pages present both on-site images, those that are store
in our image center, and off-site images, those that we request on the fly from other sites
such as CalPhotos. The following discussion only applies to on-site contributors,
such as those listed under photographers,
that are assigned an agent name and identifier.
- Agents and their database identifiers
Agents are individuals, organizations, and institutions that contribute images
and other data to Discover Life. We assign each agent database a unique identifier, such as I_SP,
which identifies Steve Paton's image database of Panamanian plant images.
- Database identifiers
Image database identifiers start with I_ (most databases) or EL_
(EcoLibrary).
Other databases, such as those containing specimen records for mapping, start with other combinations
of capital letters.
- Case sensitive
Database identifiers contain only the capital letters A-Z and the underscore '_' character.
- Unique image identifiers
- Original images
Each original image within an agent's image database is assigned a unique numeric identifier.
For orginal images, these identifiers can have up to 6 digits, each ranging from 0 to 9.
Thus, I_SP1284, is image 1284 in the I_SP database.
- Crops from original images
If an image is a crop or version of an original, it is uniquely identified by appending
an underscore followed by a unique integer to its orginal image's unique identifier.
Thus, I_SP1284_1 and I_SP1284_35 would
be versions derived from the original I_SP1284 image.
- Displaying an image and its metadata
Using the following to display this image and its metadata at the default resolution:
<a href="http://www.discoverlife.org/20/q?see=I_SP1284">
<img src="http://www.discoverlife.org/favicon.jpg" border=0> Erythrina costaricensis</a>
yields
Erythrina costaricensis
In short, use ?see=
followed by an image's unique identifier, in this case I_SP1284, to display the image and its metadata.
- Multiple images
To display multiple images together, list their unique identifiers separated by semicolons (no spaces).
Thus, I_SP1284;I_GA1643 is Steve Paton's plant image followed by ant contributed by Gary Alpert.
- Resolutions
Images are stored at five resolutions ranging from the original maximum resolution (mx)
to a thumbnail that is 80 pixels along its largest side.
The other resolutions are 240, 320, and 640 pixels in their largest dimension.
You may change the resolution and display mode of images by setting &res= as follows:
- &res=80
Displays thumbnails and their titles. Metadata is not shown unless a user clicks on a thumbnail.
- &res=240
Displays 240 pixel images with their metadata.
- &res=320
The default. Displays 320 pixel images and metadata.
- &res=640
Displays 640 pixel images and metadata.
- &res=mx
Displays images at maximum size with their metadata.
- &res=md
Switches from the default of displaying multiple images in the same page to presenting them as a slide show.
- Displaying an agent/photograph's directory of thumbnails
For each agent database, we store photographs in subdirectories that contain up to 100 original images, not including
any images made from originals and identified by the original's identifier followed by an underscore and integer.
The first directory is 0000; the second, 0001; the third, 0002, and so forth.
You can display all the thumbnails in a directory with
see=,
the agent's unique identifier, a forward slash, and the directory number. Thus
If one does not specify a directory, the first one containing an images is displayed as the default.
- Examples
You can combine the above variables:
Grading images
We give images integers as grades.
Zero is the default for ungraded images.
Negative grades switch images off so that they do not appear in species pages.
Higher grades will appear before lower ones on the species pages.
Thus, 3 is a better grade than 2, for example.
Display crops from an original
We number crops from an original with the original's unique identifier followed by an underscore and a unique integer.
For example, the Equus burchellii image I_JP2119_1 used above was cropped from I_JP2119
|
Global Mapper
The Global Mapper is provided through a technical partnership between The Polistes Corporation and Topozone.com.
Topozone provides base maps and aerial photographs from servers in Massachusetts using a total of approximately 20 terabytes of data. These
include 1:1,000,000 scale maps of the world, 17 million topographical maps to 1:25,000 scale of the United States, and aerial photographs
at 1 meter resolution covering 89% of the United States. Using Discover Life's servers, Polistes integrates distributional data onto these
from participating databases, such as Missouri Botanical Garden's. In 2003, Sun Microsystems, Inc., gave The Polistes Foundation
a generous equipment grant that included 10 processors to help Discover Life expand its network and serve its users from the University of Georgia
and Missouri Botanical Garden, in North America, and the Agricultural Research Council, in Southern Africa.
We seek additional partners to add more maps, photographs, databases, and regional servers to Discover Life.
Functions
- Help - Shows this help.
- Find place - A map will appear after you fill in the form and click on submit.
- Make custom map - Build a map of anywhere, overlaying points from participating institutions, if so desired.
Follow the instructions in red that appear on the screen.
Enter your map's center and resolution using either latitude-longitude or UTM coordinates.
The "fast" option uses cached data; the "all point" option queries the databases for their most recent updates.
- Demo - Overlays kinds from databases at the Missouri Botanical Garden, University of Georgia, and University of Illinois onto maps and images. Click on the maps and images to zoom in. Click on individual points to retrieve information about them from the respective databases.
Browser requirements
The Global Mapper and Discover Life's reporting tool require your browser to support ISMAP.
They work on all recent versions of Internet Explorer, Netscape, and Mozilla, except for Netscape version 4.79 on Mac's.
Click here for more details and how to upgrade your browser.
Put maps on your site
Because of these resources, you're welcome to put links to our global maps on your pages. The HTML examples below tell how.
Please email John Pickering
if you link your site to our maps so that you can be kept updated, if the format below should change.
- Link to a global map
<a href="http://www.discoverlife.org/20/m" target="_top">One
World</a>
- Specify image size
Adding &w=720
or 900
or 1200 specifies the map's output width in pixels. Current default is 720.
-
Link to a regional map or aerial photograph
Latitude - longitude
<a href="http://www.discoverlife.org/mp/20m?&lo=-77.03660&la=38.89760&r=.002" target="_top">White House, Our Capital, from a far</a>
Where
- &lo=longitude, in decimal degrees from the Prime Meridian, with West minus;
- &la=latitude, in decimal degrees from the Equator, with South minus;
- &r=resolution, in degrees per pixel (recommended range for satellite images: 0.5, 0.2, 0.125, 0.05, 0.025, 0.0125, 0.0083;
default = .5, the entire globe). Satellite images use tiles that are not necessarily centered on the exact point specified.
Values below &r=0.0083 center on the exact location specified.
UTM
<a href="http://www.discoverlife.org/mp/20m?&z=18&e=323387&n=4307388&r=1" target="_top">White House, Our Capital, showing aerial photograph</a>
Where z=UTM zone,
e=easting, in meters,
n=Northing, in meters,
and r=resolution, in meters per pixel (recommended values: 20.32, 10.16, 5.08, 2.54, 2, 1, 0.5; default = 2).
-
Link to a global distribution map for a kind
<a href="http://www.discoverlife.org/mp/20m?kind=Genus+species" target="_top">Genus species, Discover Life Global Map</a>
Put the particular Genus and species name in above for the kind you wish to display. Currently, through our partnership with
Missouri Botanical Garden, you can display distribution maps of many plants. As we integate additional databases, we'll
provide more species and more data for each.
Note that there is a "+" (plus sign) instead of a space in Genus+species name after kind=.
-
To map all specimens in a genus as a single color use Genus.
-
To map all species in a genus as a separate colors use Genus_ with an underscore suffix.
-
Link to a global distribution map for multiple kinds
<a href="http://www.discoverlife.org/20/q?kind=Genus+species,Genus2+species2,Genus3+species3" target="_top">Three species, Discover Life Global Map</a>
As above but separate name of each kind with comma.
-
Link to a global distribution map with renewed data -- &renew=1
<a href="http://www.discoverlife.org/20/q?renew=1&kind=Genus+species" target="_top">Renewed data for Genus species, Discover Life Global Map</a>
Add &renew=1 to the URL to force the most recent data to be loaded from the database.
However, this is slower than if you use cached data, the default.
-
Link to a global distribution map, specifying database(s)
<a href="http://www.discoverlife.org/mp/20m?kind=Genus+species;MEI;UGCA,Genus2+species2;MO" target="_top">Two species, Discover Life Global Map</a>
As above but separate name of each kind and databases with semicolons. In this case, the map would contain points from
MEI (Mandala, University of Illinois) and UGCA (Insect Diversity Project, University of Georgia) databases
for Genus species and from MO (Tropicos, Missouri Botanical Garden) for Genus2 species2. See Labels
for more details on this technology and participating databases.
Checklists
The names in a checklist may differ from those listed when you press the identify button after starting a guide.
Checklists do not list different morphs and sexes as do the guides. Checklists may include kinds that are not in the guide.
A kind must have at least one character-state attribute to appear in the guide.
Kinds that have yet to be scored and have no attributes only appear in the checklist.
Checklist are served to your browser as HTML. 20q builds HTML files from xml files. Use the following URL's to
-
View Lepidoptera checklist
http://www.discoverlife.org/20/q?act=x_checklist&guide=Butterflies
When 20q refreshes a checklist, it shows in green what xml files it included in the html file and then displays the html file.
Listserve
Thanks to Sam Droege, on 2 September, 2005, the USGS started a listserve for us to share thoughts and ask questions about IDnature guides.
It is called guidetalk.
- See archives of past discussions.
- You can join this listserve by emailing join-guidetalk@rana.er.usgs.gov
- Post messages to it by emailing guidetalk@rana.er.usgs.gov
|
Import -- how to get information from your databases into our tools
Our goal is to enable you to use Discover Life's tools to process and share your information.
The IDnature Guides (20q) and Record Manager (20l) have browser interfaces that enable contributors
to enter and manage data manually via the Web.
Here we discuss how to get information from databases into our tools.
We consider
|
File format
We import images, text tables, XML files, and HTML files using various utilities into our tools.
Here we consider the simple case of importing tables. For more complicated cases, please contact
our support center.
Importing tables
Our IDnature Guide, Image Center, Record Manager tools import ASCII text tables.
They give data providers maximum flexibility and do not force them to use a rigorous data schema.
Providers name and order fields that they wish to share. The tools index key fields,
such as scientific name and geographic coordinates. They pass geographic information to the Global Mapper,
which in turn combines data from all contributors into maps.
They also integrate the information into the species pages served by the Search Box.
Here we describe how to format text tables from Excel, Access, and other databases so that we can import them.
Each table requires the following format:
- Header
The first line (row) must be a header that
describes the fields (columns) in the table's subsequent lines.
- Field separators
Separate fields with either tabs or dollar signs.
If you are exporting data from an Excel worksheet, for example,
use the "Tab delimited" option under "Save as" to create the table as a text file.
- Field names
Name the fields anything that you want.
For example,
"Unique identifier$Scientific name$Latitude$Longitude"
is a header that describes four columns separated by dollar signs. And,
"id$title"
describes a table with just two columns. Within reason, you can have as many columns as you wish.
The software recognizes the following special syntax:
- Names ending in 'URL' allow you to put links on pages.
- Names beginning with '_' (underscore) are typically hidden from the end-user.
- Names ending with ';' (semicolon) can have subfields that are separated by semicolons.
- Names beginning with 'CS:' (CScolon) allow you to import character-states into IDnature guides.
- Field order
Order the fields as you want them to appear in subsequent lines in Web pages.
- Record separator
End each line with either a carrage return, a line feed, or both.
- Records
Follow the header line with one or more data records (= lines = rows).
Each data record must have the same number of fields, in the same order, as the header.
Thus, if a record has missing values, include nothing between the field separators.
- Unique identifiers
We use unique record identifiers to retrieve data. If your database has unique
identifiers, include them in a column. If not, the system assigns them sequentially,
starting with the first record after the header. We create identifiers
that are globally unique. We do this by combining a prefix, that is unique to each database,
with integers, that are unique within each database. For example, KSEM is
a prefix that is unique to Kansas Entomology Museum. KSEM1000 is unique record, number one
thousand, within this database. Within a table to be imported, the agent's prefix is optional
and not required in unique identifier column. Thus, in this case, "KSEM1000" or just "1000"
are treated the same. Leading zeroes have meaning. "KSEM01" is not the same as "KSEM1".
- Flat versus relational data
While it is much simpler to import data as a single flat file, our Record Manager can import
relational data from multiple tables. One can do this by including a field at the end of
a header line that links records to records in subsequent tables. For example, imagine
that you have a relational database that contains two tables,
SPECIMEN.txt and COLLECTION.txt, in which
records in SPECIMEN.txt may point to data in COLLECTION.txt.
In this case, add a field at the end of the header line of SPECIMEN.txt called "COLLECTION.db".
Link specimen records to their collect record by putting the collection record's unique identifier in
the COLLECTION.db field. This is shown in the MEI example below.
An example if a table with 4 columns that would plot two Quercus points in California:
___________________________
id$name$Latitude$Longitude
1$Quercus$37.8$-122.4
2$Quercus$33.5$-117.8
___________________________
An 8 column example:
___________________________
id$title$Latitude$Longitude$state$county$city$SourceURL^M
1$Golden Gate Bridge$37.81103$-122.47788$California$SanFrancisco$SanFrancisco$http://www.discoverlife.org Discover Life^M
2$Cresent Bay$33.54690$-117.80201$California$$Laguna Beach$http://whatever2...html Source^M
___________________________
Here we use Control-M to indicate the end of lines.
Note $$ in id=2, as no county for Long Beach is given.
Note the space after 'html' and before 'Source'.
Our tools treat certain fields in special ways.
The following are used by the
- Record Manager -- to import specimen records for labelling and mapping
- id -- unique identifier, as described above
- name -- scientific name, such as a genus and species, genus, or family.
- authority -- author and year
- geographic coordinates -- we support a large number of formats
that support importing latitude-longitude and UTM coordinates. Ideally,
we want two fields, "Decimal latitude" and "Decimal longitude,"
where South and West are negative values.
Examples:
- Missouri Botanical Garden's Tropicos database uses one header line:
id$Scientific Name$Authors$Country$State/Province$County$Locality$Elevation$Coordinates$Date$Senior collector$Collectors$collection number$Herbarium
where id='id', name='Scientific Name', authority='Authors',
and geographic coordinates are in 'Coordinates'.
See output example -- MO00123456.
- National Science Foundation PEET Therevidae database uses the specimen table header:
SpecimenID$TaxonAuYr$CollObservations$LocalityLabel$Determination$CurrentDeposition$StagesSexTypeCondition$PreservationState$Dissection$GenbankACC$GenbankGene$GenbankURL$SpecimenNotes$COLLECTION.db
that relates to a collection table with the header:
Parent_ID$BiogeographicRegion$Country$StateProv$LocalityBelowState$Latitude$Longitude$ElevationRange$TrapSite$CollectingMethod$Collectors$CollDateRange$Conditions
where id='SpecimenID', name+authority='TaxonAuYr',
and geographic coordinates are in 'Latitude' & 'Longitude'.
See output example -- MEI164494.
- Academy of Natural Sciences, Philadelphia, uses one header:
taxonname latitude longitude depthrange year month day source collectioncode institutioncode phylum kingdom termscitationURL basisofrecord
where id is assigned by system, name='taxonname', authority is unassigned,
and geographic coordinates are in 'latitude' & 'longitude'.
See output example -- ANSP12345.
- Image Center -- to import image metadata into EL_DP and I_* databases
- id -- unique identifier
- title -- an image title. Start with scientific name, if known.
- _grade (note leading underscore) -- use integers to grade images as as follows:
- >0 -- images with higher intergers appear before ones with lower grades
- 0 or blank value -- ungraded images will appear last
- <0 -- images with negative grade will not appear on species pages but may be accessed through the Image Center's tools
- caption -- text that appears below the image; may include HTML code; cannot include end-of-line characters
- IDnature Guides -- to import names, authorities, and attributes into checklists and guides
- path -- Row value for an ant, for example, is 'Insecta/Hymenoptera/Formicidae'
- name -- Row values are specific 'Genus species', or 'Genus', or 'Family', ...
- authority -- Row values are 'Smith', 'Smith, 2005', '(Smith, F., 1900), etc.
- common_name -- To import common names and synonyms; separate mutitiple ones with semicolons; use unicode
symbols to represent apostrophies (i. e. use ' not ') and accents
(see list of unicodes for accents).
- level -- Row values are 'species', 'genus', 'family', etc.
- links -- to import links (for format, use 'Score' and submit in a guides 'Menu' and read red text).
- notes -- to import notes.
- CS:Whatever character -- Row values are 'State' or 'State1;State2' in the case of polymophisms.
Data transfer
We use the following options to transfer data:
- Web files
Our servers can get files from the Web. Ideally, we want data providers to
put text and image files on the Web so that our servers can copy them,
index their information, integrate it with information from other sources,
and then mirror it to end-users via our tools. For active databases,
we set up automated nightly processes that check files on contributing Websites and
update our servers.
As a simple example, please export your spreadsheet to a text file, put it on a
Website, and keep it updated. We just need to know it's URL. Our servers will do the rest.
- Web services
This is a more sophisticated option in which our servers query Web-enabled databases for information.
It is too complicated to cover here.
- User accounts
For data providers who have Internet access but who don't have their own Website,
we provide user accounts on Discover Life servers that allow you to transfer images
and text files to us via ssh/scp protocol.
- Other means
Provides can send us data as text within emails, as email attachments, on disks, and by other means.
This is fine if the data are stable. However, it is a pain each time you update it.
Setting up new databases and automating updates
It is easy to import images and data into Discover Life. (Yeah, sure. Ha! Ha!)
This section is to help system operators working at Discover Life centers.
While not quite black magic, it's not for data users, data providers, or the faint at heart.
But no worries. By the time we've finished this documentation, the process may be automated and our
operators out of a job.
Table name or URL:
If you have a single file to import, call the file $agent.txt, where $agent is your organization's unique code.
$agents must be named with capital letters and no spaces; usually abbreviation of institution or database compiler
For example:
EL_DP = EcoLibrary, Dan Perlman
ASJ = Airlan San Juan
I_SB = Images from Steve Baskauf
Thus, I_SB.txt is a valid filename.
This file can be put anywhere on the open Web.
Alternatively, you can request an account on Discover Life and put it in your top directory.
Table format:
You must format tables as a text file with either tab or dollar signs separating fields.
For details on formatting the file and the fields that it must contain see File format above.
Crediting images and records:
Create a file called /home/IM/db/$agent/README to credit images and allow users to send feedback to data providers.
The README file has the following lines:
-
email: who@server.domain (the email account for users to send feedback)
-
copyright: http://URL Text credit (the URL of the copyright statement, followed by a space, and the text credit)
-
logo: HTML code to embed a logo and links above displayed records.
For example see KSEM's README.
Formatting images to import:
Images must be formatted as a JPEG with file extension being in lowercase and the filename containing the $agent and a unique identifier (eg. I_SB98.jpg). If you can't rename your images with the $agent, we can do that for you, but you will have to assign a unique identifier to each image in the file name.
Steps are:
- Make the following subdirectories changing the permissions as stated:
- /home/IM/$agent
- chown polistes $agent
- chgrp nobody $agent
- chmod 775 $agent
- /home/IM/$agent/programs
- chown polistes programs
- chgrp nobody programs
- chmod 750 programs
- Editing agents file
- vi /home/IM/db/agents
- use header as a guide to add additional line for new agent
- agents contains a list of agents and corresponding databases
- the first 6 fields are required, remaining ($F_authority, $F_collection_id, $F_la, $F_lo) fields not necessary to run program
- Creating nightly program
- Copy /usr/local/sbin/nightly.pick5 into /home/IM/$agent/programs/nightly leaving out the other databases' command lines
- add job to nightly.pick5 only if it is required to run every night
- nightly should contain: (text in red is only required for image databases that have a corresponding guide or .xml file)
#!/bin/sh
echo "START**************************`date`**********************************"
#Note new program
/home/IM/db/programs/web2db -w $agent #for I_RR: add -i to re-process images
chown -R polistes /home/IM/db/$agent
chgrp -R nobody /home/IM/db/$agent
chown -R polistes /home/IM/$agent
chgrp -R nobody /home/IM/$agent
/usr/local/bin/ssh root@www.discoverlife.org '(cd /home/web/dl/nh/id/20q; /usr/local/bin/bak EcoLibrary.xml)'
/usr/local/sbin/scrape_url 'http://pick5.pick.uga.edu/mp/20q?act=x_import&email=perlman@brandeis.edu&import_url=http://www.discoverlife.org/im/db/I_SB/ID_parent.txt&guide=EcoLibrary&flags=import_no_db:'
rm 'http:::pick5.pick.uga.edu:mp:20q?act=x_import&email=perlman@brandeis.edu&import_url=http:::www.discoverlife.org:im:db:I_SB:ID_parent.txt&guide=EcoLibrary&flags=import_no_db:'
/usr/local/bin/ssh root@www.discoverlife.org '(cd /home/web/dl/nh/id/20q; mv EcoLibrary.xml3 EcoLibrary.xml)'
/usr/local/sbin/scrape_url 'http://pick5.pick.uga.edu/mp/20q?act=x_refresh&flags=240_no:&email=perlman@brandeis.edu&guide=EcoLibrary'
rm 'http:::pick5.pick.uga.edu:mp:20q?act=x_refresh&flags=240_no:&email=perlman@brandeis.edu&guide=EcoLibrary'
echo "STOP**************************`date`**********************************"
- Creating your .org file
- For all databases:
- you need /home/IM/db/$agent/$agent.org
- this file is created first -- done with first command in nightly /home/IM/db/programs/web2db -w $agent, which runs the web2db program
- When running the web2db program to create the database, the program will ask you to:
As root, chgrp nobody /home/IM/db/$agent
- then check in the labeling system to make sure it build the database
|
Scientific names
Discover Life and its partners now manage over 1.18 million valid scientific names on-line.
These are associated with over 10,000 family names that are part of our
Tree of Life,
which is based on The National Science Foundation's Assembling the Tree of Life project.
They can also be accessed via our search box
and by browsing All Living Things.
We encourage taxonomists to contact us and use our tools to contribute and manage their scientific names,
authorities, synonyms, references, and associated biological information.
Using the same valid scientific name across databases is essential to our collective efforts to link checklists,
identification guides, photographs,
catalogs, specimen records, maps, and other information into pages on species and higher taxa. Here are the rules that
we follow in sharing names across databases.
- NO ACCENTS -- Letters in scientific names are restricted to Latin A-Z and a-z.
Although we support accents in authority names, references, and elsewhere on Discover Life
(see accents & symbols),
our scientific names never have accented characters.
- DATA STOREAGE VARIABLES -- We use up to three variables to STORE valid scientific names.
These variables are $name, $morph, and $sex.
We use an additional variable, $common_name, to store other names,
such as synonyms, alternative spellings, and vernacular names in different languages.
We separate multiple synonyms and other names with semicolons within the $common_name variable.
We use the variable $authority to STORE the author (and possible year and page combination)
that ties a $name to its scientific reference. We use a final variable, $level, to specify the level
of a $name, e. g., species, genus, and family.
- IMPORT VARIABLES -- To support the vast array of non-standard taxonomic databases, we do not require
contributors to have their data in any particular format. We have a large number of translators that
IMPORT data into the storage variables described above. For example, many taxonomists keep $genus and $species
variables as separate columns in their spreadsheet and database tables. For species level names,
our translators combine $genus and $species and store the binomial in $name,
separating the genus and species with a space.
- VALID HIGHER TAXA -- Valid genera and higher taxa always start with a capital A-Z
and have one or more lowercase letters a-z.
They contain no spaces or accents and are stored in $name.
- NEW GENERA -- Unpublished, new genera start with the convention 'Gn_' (capital-G lowercase-n underscore)
and then follow the rules of morphospecies below that try to separate them to a unique, temporary generic name.
- VALID SPECIES -- A valid species name is a binomial with a valid genus and specific name. The specific name is
all lowercase a-z but may include a hyphen '-', e. g., 'Smilax bona-nox'. There is a single space between the
two names. It is stored in $name.
- MORPHOSPECIES -- Morphospecies names are temporary, working names that are stored as binomials in $name.
The second, specific name must start with 'sp_' (lowercase-s lowercase-p underscore) and be followed
by a unique investigator-place-project-number combination that tries to distiguish it from other morphospecies
but may fail to do so across databases, e. g., 'Rogas sp_Panama_3' or 'Rogas sp_UGCA_5'.
Use underscores, not spaces to separate words.
Put 'sp_' before 'cf', 'nr', 'n_sp', 'near' and other ways or describing morphospecies, e. g.,
use 'Jornandes sp_n_sp_from_chiapas' or 'Scalponotatus sp_nr_chilcotti_2'.
We convert 'sp.' & 'spp.' either to just the genus name or to a morphospecies, e. g., sp_1 versus sp_2.
- SUBSPECIES, VARIETIES, FORMA, AND HYBRIDS --
The variable $name links to a web page. It CANNOT be a trinomial with two spaces.
You have two options to store information about subspecies, varieties and forma.
- Option-1 [preferred] -- use the $morph variable
If we import the name of a subspecies, variety, etc. into the $morph variable,
then it will be associated with the species page to which it is a member.
There are on-line tools within the IDnature guides for managing information associated with
the $morph and $sex variables.
- Option-2 -- append the third name after an underscore to the end of the $name variable
This option separates the kind from its species and gives it a page under the genus.
- Subspecies: use 'Genus species_subspecies', e. g., 'Wasmannia auropunctata_australis'.
- Variety: use 'Genus alpha_var_beta'.
- Forma: use 'Genus alpha_forma_beta'.
- Hybrids: use 'Genus alpha_x_beta'.
- HOMONYMS -- In the case of homonyms, as a temporary measure,
we append '_homonym' to one $name to distiguish it from another taxon with the same name,
e. g., Cepolidae and Cepolidae_homonym, which we use as fish and gastropod family names, respectively.
Use '_homonym1', etc., if necessary.
- TAXONOMIC AUTHORITY FILES -- Our taxonomic authority files associate species with higher taxa
and take precedence over other information on names. Contributing taxonomists can import and
manage these files with the IDnature guide (20q) software. The databases that manage images and maps use these
authority files to associate species with higher taxa. They do not contribute to the authority files.
BE WARNED that if a species is in a map database, it will NOT map as a member of a genus unless the species is
also a member of the genus within an authority file. While this may seem inconvenient, it keeps a large number
of spelling errors within map and image databases from creating bogus species within genera.
- IMAGE TITLE -- We use an image 'title' variable to associate images with species pages. To do this, the title value
should start with a valid scientific name. It may have additional information after a comma. For example,
'Solenopsis invicata, queen, head' will associate the image with Solenopsis invicta's species page,
assuming that 'Solenopsis invicta' is in an authority file!
|
Labels
20q links to a labelling sytem, 20l. As of 24 February, 2003, this labelling system uses the following XML schema:
<parent id="$id" gmt="$when_gmt" ip="$ip">
<latitude>
$la
</latitude><longitude>
$lo
</longitude><accuracy>
$accuracy
</accuracy><elevation>
$elevation
</elevation>
<country>
$country
</country><state>
$state
</state><county>
$county
</county>
<city>
$city
</city><site>
$site
</site><position>
$position
</position>
<time1>
$time1
</time1><time2>
$time2
</time2><days>
$days
</days>
<who>
$who
</who><email>
$email
</email>
<how0>
$how0
</how0><how1>
$how1
</how1><how2>
$how2
</how2>
<how3>
$how3
</how3><how4>
$how4
</how4>
<habitat>
$habitat
</habitat><field_note>
$field_note
</field_note>
<note>
$note
</note>
</parent>
|
Browsers
Discover Life's 20q software supports most recent browsers,
including Internet Explorer version 4, Netscape version 6, and Mozilla version 1.0.2.
There are a few bugs in
recent versions of Internet Explorer (version 5 needs Flash) and Netscape (version 6 under Mac OS X) that cause some images not
to appear in full color or at high resolutions. If you have problems, try using IE 4, which works on Windows and Mac platforms.
Netscape 4 has problems with our interactive maps on Macs but version 4.7.6 works under Windows. Map targets do not display on any IE
or Netscape version 3 or below.
If you need to upgrade your browser,
click here to upgrade to Mozilla or
Firefox.
|
|
File transfer -- download ssh & scp
If you are a data provider with an account on Discover Life, you will need ssh/scp communications software
to login to our system (pick2.pick.uga.edu or pick9.pick.uga.edu) and transfer files.
ssh lets you run commands on Discover Life servers. scp allows you to transfer files back and forth.
For security reasons, we no longer support older protocols such as telnet and ftp.
- Mac OS 10 users
- ssh
Use the ssh command from within your "terminal" application.
- scp
We customize two shell script programs, dlup and dldown, that Mac users can run via their "terminal" application
to transfer files between a DiscoverLife folder on their desktop and their account on Discover Life's
pick2.pick.uga.edu or pick9.pick.uga.edu servers.
For uploading images, goto
Learn how to upload pictures from a MAC OS X
- Linux and Unix users
Use the native ssh and scp commands that are included with your operating system.
- Windows users
We recommend that you download
SSHSecureShellClient from Discover Life
or from www.ssh.com.
This program includes both ssh and scp features and is more secure than using the following alternatives:
-
a ssh terminal program, putty.exe (to download click on putty.exe)
-
a scp file copy program, winscp (to download winscp, click on
winscp)
For uploading images with winscp, goto
Learn how to upload pictures from a PC
|
Disclaimer
We make no guarantees to the accuracy of IDnature guides. If you find errors, please report them to us.
Contact us
Please send your suggestions of how this software and guide can be improved to
John Pickering.