2.1. Installing a Subversion client

How you will do this depends on which operating system you are using.

2.2. Checking out the repository for the first time

Once your subversion client is installed, the first thing we need to do is to check out the repository. To do this, you need to open a terminal window:

• On Windows, click on the Start menu and type cmd into the search box.
• On a Mac, select Terminal from the Utilities folder in Applications.
• On Linux, press Control + Alt + T.

Now we'll check that svn is installed and working. Type svn checkout and press return. If the terminal responses that there are Not enough arguments provided, then svn is working OK.

And now we'll check out the files from the repository into our directory. Make sure you don't forget the space and period at the end of this command:

svn checkout https://hcmc.uvic.ca/svn/dvpp⚓

This should create a new folder called dvpp and all the files from the repository should appear in it.

2.3. Using SVN in your daily work

These are the basic rules when you're working with svn:

Before you start work, update your local files:

• Open a terminal

• cd dvpp⚓

• svn update⚓

You can leave the terminal open while you work. Then, when you're ready to commit your changes:

• svn update⚓

  (again, just in case anything else has been committed by someone else)

• svn commit xml -m "A message explaining the changes you have made"⚓

(Aside: we include ‘xml’ after the word ‘commit’ to specify that we only want to commit any changes that are in the xml folder. That way, if we've accidentally made changes to program code or other files in the repo, those changes won't be committed.)

That's basically it. If you see any warnings or error messages from svn, check that you're in the right folder in your terminal. You may also see error messages if two people have been editing the same file at the same time, and Subversion needs you to make a decision about whose changes should be kept.

2.4. Tips for using the terminal

Finding out where you are right now on Linux/Mac:

pwd⚓

On Windows:

echo %cd%⚓

Moving up the tree one level (Windows, Mac and Linux):

cd ../⚓

List files in a folder on Linux/Mac:

ls⚓

and on Windows:

dir⚓

List files in a folder with full information:

ls -al⚓

3.1. Oxygen code templates

Our Oxygen project file has a number of useful keyboard shortcuts built into it, which you can use to speed up your encoding. Generally speaking, to use these, you select some text in the editor, then press the keystroke shortcut shown in the table below. The table uses the following key names:

3.2. Using ‘QuickFix’ in Oxygen

We're able to make use of the Schematron QuickFix functionality supported by Oxygen to provide some convenient encoding shortcuts. QuickFixes are listed and described below.

Basically, the way a QuickFix works is that you do something which contravenes a Schematron rule; this causes you to see a warning or an error. Then you right-click (or Control-click) on the element which is showing the error, and you will see the option to ‘Quick Fix’ in the popup menu. Click on this, and then choose the QuickFix you want to apply (usually there is only one). You can also press Alt (or Option) + 1 (the digit one) to apply a QuickFix.

4.1. Part 1: Person Research and Person Tagging on XML Poem Files

Initial metadata collection is done using a spreadsheet in MS Teams. After poems have been proofed by Alison in the spreadsheet, batches of volumes are transformed into XML files by Martin. Each XML poem record needs to be tagged with the correct person id, as applicable, for poet, translator, and illustrator.

4.2. Part 2: Person Record Editing Guidelines

The Prosopography (historical personography) is stored in the xml/prosopography folder, and is divided into files by alphabetical letter (everyone whose surname begins with A appears in the prosopography_a.xml file). If you are researching the author, translator, or illustrator of a poem, and you cannot find them in the prosopography, then you will probably have to add a new entry. Note that each individual has only one record, whether they are a poet, a translator, and illustrator, or any combination of these. This is an example of a very rich and full person record, relating to a real person whose identity is known:

NOTE: ashb2 is the next available id for the letter prefix ashb.⚓

The sex attribute is an impoverished method of recording what appears to be the assigned gender of the person; here we use the values from the ISO/IEC 5218 specification:

• 0 = Not known;
• 1 = Male;
• 2 = Female;
• 9 = Not applicable.

Next is the <persName>, or personal name, which contains <surname> and <forename> elements, along with a <name> type=displayName. This is what will be used for display on DVPP web pages which mention the person. It should be the most common name used by the poet/illustrator as a signature, or by which the poet is known historically, the form of [SURNAME], [FORENAMES]. If a person is usually or commonly known by their pseudonym, add it at the end of the display name in parentheses, e.g. Evans, Marian (pseudonym: “George Eliot”). Surname/First Names: the full historical names of the author/illustrator.

If there are other people with the same name, add the birth and death dates (where known) to the display name. Where dates aren’t known, add another distinguishing feature (e.g. there are two John Browns, neither of whom have known dates, so one has the display name Brown, John (Corporal). 

If you can ascertain the nationality of the person, include it in a <nationality> element. Don’t add British as a nationality unless you really don’t know if they’re English, Scottish, Irish, or Welsh. Terms for nationality like Anglo-Irish, Anglo-French etc. are also used sparingly.

If you can find the person in VIAF, you can include their VIAF number using an <idno>tag.

Finally, you will need to add the <note>, which contains short biographical note along with any other information that you need to specify relating to uncertainty, ambiguity, and so on.

At the other end of the scale from the rich record above, there will be many cases where we know little or nothing about the poet, and a very limited record will be all that you can create:

In these cases:

• Do not assume any metadata from their fictitious name. E.g ‘An English Lady’ is not necessarily English nor a Lady. The sex attribute will be 0 (not known).
• Omit any elements which cannot be completed.
• Initialised pseudonyms are given in the display name field as printed on the poem (e.g.

  A. B. C.⚓

  not

  C., A. B.⚓

  ) Also, for poets and translators add the periodical and role, and for illustrators just add the role (e.g.

  A. B. C. (poet; Argosy)⚓

  ,

  A. B. C. (translator, Argosy)⚓

  ,

  A. B. C. (illustrator)⚓

  .
• If a poet is given as ‘By the Author of xxx’: can you find the name of the author obviously, such as through the periodical volume’s contents/index or by a quick Google search? If the real name isn’t obvious, give ‘The Author of xxx’ as the display name.

7.1. The title statement

Poem signed “#.”⚓

7.2. The publication statement

Every TEI file should include information which tells any potential reader or user of the file who created it and who owns it, and also provides some guidelines as to what kind of use is permitted. The information in our files is currently provisional, because we have not discussed licensing yet. When decisions are made, all publication statements in all files will be updated (or more likely linked to a single central block of information in another file).

7.3. The source description

Information about the original source publication is provided in <teiHeader>/<fileDesc>/<sourceDesc>. Here is an example:

You should not directly change any of this information, since it comes from the database.

7.4. The encoding description

We currently only create or edit <encodingDesc> when doing XSS styling; see Avoiding repetitive CSS below. More information on encodingDesc will be provided in the future.

7.5. The profile description

In the <teiHeader>/<profileDesc>/<textClass> element, we provide a set of links to a range of different taxonomies that the project is developing for the description of the form, structure, and topics of the poems. Here is an example:

These <catRef> elements have only one piece of information, in the target attribute: a pointer to a category which is defined in our taxonomies.xml file. You can look at the taxonomies.xml file to see how that works, and there is a rendering of the complete set of taxonomies below but the basic idea is that there are several taxonomies (for genre, layout, authorship etc.). Each of these taxonomies has a hierarchical structure, so for instance there is a category genreLyric, and that contains genreLyricSonnet, which in turn contains genreLyricSonnetPetrarchan. When adding <catRef>s to a file, you need only add the longest (i.e. the deepest-nested) one in the tree which applies; if you add e.g. genreLyricSonnetPetrarchan, you don't need to add genreLyric or genreLyricSonnet, because these are inherited.

Through an automated process, the taxonomies are incorporated into the project ODD file and schema, so when you add a <catRef> element and start typing its target attribute, Oxygen will prompt you with a complete list of all the available values, along with descriptions of what they mean. Try to add at least one <catRef> from each taxonomy.

Ultimately, these categories will be used to create a sophisticated search system which allows users to retrieve poems based on a wide variety of different features and characteristics.

7.6. The revision description

The <revisionDesc> element is the last component of the <teiHeader> element. It is optional, but until it is added, no document can be published. It has a mandatory attribute status, whose values can be generated, encoded, proofed, and published, and a mandatory child element <change>.

Note that the word ‘proofed’, which is equal to the value of status, appears in the text of the <change> element. This is enforced by Schematron, to ensure that any change to the status value has a corresponding <change> element to match it.

When there is no <revisionDesc> in a document, it is presumed to be either empty (no transcription or encoding), or, if it has an encoded poem, it is presumed to have the encoded status. Any document with the proofed status is ready for checking by Alison, prior to publication; any document with the published status will be shown on the next release of the website as a completed text. Unpublished documents will still appear, but will have a clear signal that they are not yet formally published.

7.7. Taxonomies and categories

This is the current set of taxonomies and categories in the project, taken from the taxonomies.xml file. These are used for various purposes, including the classification of documents using <catRef>, and the generation of value lists for attributes.

9.1. The front matter

Any front matter for any individual poem will be created by you, the transcriber/editor/encoder. It may consist of a brief introduction to the poem, in the form of one or two paragraphs, or it might be a complete critical introduction on a much larger scale. It should be encoded in a <div> element, possibly with nested <div>s if it has multiple sections, and should consist of the usual TEI prose structures (<head>s, <p>s, <quote>s, etc.)

9.2. The back matter

The back matter will consist of any textual notes or other information that does not belong in the body of the document. However, note that you should not put footnotes in the back matter; a footnote is a TEI <note> element, which should be included in full at exactly the point in the text where you would expect to see the footnote number in a print document. See 15.2. Editorial notes for more information.

10.1. The main poem <div>

A single <div> with type=poem should enclose the entire work. If it is a cycle of poems, use type=cycle. You can use the style attribute on this <div> to describe the space within which the poem is printed. Normally, this will be a page or a column. For instance, a poem printed within a column might be encoded like this:

This specifies that the column within which the poem was printed is about 23ems wide.

10.2. Titles and headings (<head>)

The title of the poem (if there is one) will go inside a <head> element in the main poem <div>. If the poem is divided into sections with their own headings, each one of these sections will constitute a <lg> (line-group) element, and the section heading will go in a <head> element at the beginning of the <lg>:

Poems in a cycle are treated as a single text for the purposes of encoding, which means that in addition to appearing in the same file, they also share the same rhyme labelling sequence. So if the labels a, b and c are used in the first poem, the second poem will start its rhyme labelling from d.

10.3. Prose components

Some poems will be preceded by, or will include, some explanatory prose such as an introduction or a letter. There are two ways to encode this material, depending on where it appears.

10.4. Line groups and lines

We normally think of line-groups (<lg>) as stanzas, but we should actually encode all convenient or useful groupings of lines, both larger and smaller than actual stanzas, as line groups. For instance, we might encode a Shakespearean stanza like this:

Note that this implies some degree of interpretation of the poem; some scholars might enclose the quatrains in a line-group on the same level as the couplet; others might divide a sonnet into an eight-line group and a four-line group.

10.5. Broken lines

10.6. Refrains and other sonic devices

We use the term "sonic device" to refer to a number of rhetorical devices in which words, phrases or whole lines are repeated for effect.

10.7. Encoding rhyme

In this project, we encode two different aspects of rhyme: the rhyme-scheme pertaining to a line-group, and the actual rhyming words or syllables that constitute each individual rhyme. Rhyme-scheme is encoded using the rhyme attribute:

Where a line in the structure does not rhyme with anything else, use a dash (hyphen) instead of a letter of the alphabet.

Rhyming words or components of words are tagged with the <rhyme> element:

The letter in the label attribute should match a letter in the rhyme attribute on the containing <lg> element. The type attribute has a limited set of values representing types of rhyme; these are documented in the taxonomy and included in the schema.

 Here are some rhymes
 That happen three times
 In only three lines.
 
 The first is the rule;
 The second one's cool,
 But the third one is vile.
 
 If there are only two,
 The first can't be wrong cos it’s new,
 So the second gets tagged as half,
 But only the second, not both.⚓

*dis | play⚓

10.8. Page breaks and column breaks

We would like to be able to retrieve poems based on the number of pages or columns they span, so it's important to encode this information. Whenever a page or column break appears in the text, mark it with a <pb> or <cb> tag. This actually includes the beginning of the poem itself: we need to signal that the poem begins on a page, or (if there are columns) in a column, so the first thing in your encoding will be a <pb> tag, and if there are columns, there will need to be an appropriate number of <cb> tags too. Here's an example of a poem that begins on the second column of a page, and runs over to the next page:

Note that you will sometimes encounter mixed-column layouts, in which parts of the poem span multiple columns while other parts respect the column divisions. See Mixed column layouts for detailed guidelines for such cases.

10.9. Bylines

Many poems have bylines at the head or foot of them, like this:

Here there are two byline blocks, one right-aligned, containing the initials of the poet, and one left-aligned, containing the place and date of composition. We will tag these as separate <byline> elements for convenience:

<byline style="text-align: right; margin-right: 3rem;">  <persName>E. P.</persName> </byline> <byline style="margin-left: 2rem;">  <placeName>Paisley</placeName>, <date when="1840-02-26">26th Feb. 1840</date>. </byline>

Note that this project’s use of <byline> is very broad; we do not distinguish between datelines and bylines, but instead we use <byline> for any such component. Within bylines, though, we tag <persName>, <placeName> and <date>.

Sometimes a byline will include the life-dates for an author (birth and death dates). In this case, do not tag these as dates; we only tag a date of composition in as a date in the byline.

10.10. Closed (fixed) form

Poems with a closed (or fixed) form follow strict patterns that require specific handling when encoding. These poems will sometimes be separated into stanzas, but they will need a parent <lg> element with an rhyme attribute that surrounds and includes the entire poem. Sometimes, these poems will have their closed form in the title; however, this is not always the case. Closed forms to watch out for include:

• Ballade: Features an envoi (usually as an address). Usually 3 octaves and a 4-line envoi. The last line of the first stanza is usually the refrain, repeated in the last line of each stanza and the envoi. Octaves usually rhyme "ababbcbC", and envoi "bcbC". Ballade example.
• Rondeau: ​12-line or 15-line poem that features 2 rhymes. Usually the first word or phrase of the first line is the refrain (the rentrement) and does not rhyme. Often printed in 2 or 3 stanzas (often 5 lines, 4 lines, 6 lines). Example rhyme scheme (but varies): "aabba aabR aabbaR" (R=refrain). Rondeau example.
• ​Rondel: Usually a 14-line poem (sometimes 13-line) with 2 rhymes ( 2-line refrain that repeats 2 or 3 times). Example of rhyme scheme: "ABba abAB abbaA(B)". Note: sometimes looks like (but is not) a sonnet. Rondel example.
• Sonnet: Often printed as one line group, but sometimes there is a gap between the octave and the sestet. When there is not a space, nested line groups still need to be added around the octave and the sestet or the quatrains and couplet. Some example patterns are “abab cdcd efef gg” (Shakespearean) and “abbaabba cdecde" (or "cdcdcd)” (Petrarchan). Sonnet example.
• Triolet: ​8-line poem with 2 rhymes and 2 refrain lines. Pattern: "ABaAabAB." Triolet example.
• Villanelle: Usually a 19-line poem with 2 rhymes and 2 refrains. Usually "aba" stanzas and final stanza "abaa." Usually the first and third lines of the first stanza are repeated alternatively throughout the poem and then end up as the final two lines of last stanza. Villanelle example.

10.11. Verse dramas

Verse dramas require specific encoding that differs from our usual poems. The outer <div> type attribute should have the drama value, and for verse dramas made up of various acts and scenes, multiple nested <div> elements with act and scene values should be used.

We use <speaker> tags to surround the speakers name and <sp> tags to surround <speaker> and <l> tags.

Sometimes verse dramas will include a cast list. Each <castList> is made up of <castItem>s, which can include <role>s (the role title or character name) and <roleDesc>s (the description of that role), when present. Sometimes cast lists will include a <head>.

We tag stage directions with <stage>. Each <stage> tag must have at least one type that indicates the type of stage direction, but more than one can be selected when a single stage direction performs multiple functions. The type values, with examples, are as follows:

• Business: describes stage business.
  <stage type="business">Pointing to his breast.</stage>
• Entrance: describes an entrance.
  <stage type="entrance">PLUTO enters.</stage>
• Exit: describes an exit.
  <stage type="exit">Exit the Master of the Hospital.</stage>
• Delivery: describes how a character speaks
  <sp>  <speaker>Soph.</speaker>  <l>   <stage type="delivery">(Weeping)</stage>Alas ! my daughter !</l> </sp>
  <sp>  <speaker>Wal.</speaker>  <l>   <stage type="delivery">aside.</stage>  </l> </sp>
• Location: describes a location.
• Modifier: gives some detail about a character.
• Novelistic: is a narrative, motivating stage direction.
• Setting: describes a setting.
  <stage type="setting">Scene, Drury-Lane Theatre — Time, Midnight.</stage>

In the above examples, each stage direction performs one function. In the following example, the stage direction performs multiple functions, so more than one value is applied:

11.1. Illustrated poems

11.2. Ornaments

Ornaments are features such as horizontal lines and decorative figures which enhance the text but do not form part of it.

11.3. Marginal braces

Sometimes you may see a number of lines which are distinguished by having a marginal brace, typically a curly brace in the right margin. This is usually done to signal when a rhyming triplet appears in the context of a sequence of couplets.

11.4. Drop-capitals

NOTE: This is a crude approach to a potentially very complex problem, so if you're having trouble making it work, talk to the programmer.

11.5. Epigraphs

Sometimes there will be more than one epigraph at the beginning of a poem. In these cases, each are tagged separately with their own <epigraph>, <quote>, and <bibl> tags.

11.6. Foreign Languages

We use <foreign> when some text appears stylistically different from the surrounding text (typically in italics), and the apparent reason is that the text is in a language other than English. Non-English text often appears in epigraphs.

In these cases, it is good practice to tag the text with <foreign> instead of <emph> or <hi>. When using <foreign>, add the global attribute xml:lang and use a language subtag as the value (see the IANA Language Subtag Registry). This encoding will enable our DVPP language consultants to isolate, translate, and when needed, transcribe non-English text.

The following language subtags are common in our project.

• la Latin
• grc Ancient Greek
• el Modern Greek
• fr French
• it Italian
• de German
• es Spanish
• gd Scottish Gaelic
• tr Turkish

11.7. Compositor errors

Occasionally, you will come across situations where the printer has made an error. The original error should be transcribed accurately, but you should tag it and supply a correction using the elements <choice>, <sic> and <corr>:

<choice>  <sic>error</sic>  <corr resp="dvpp:id_of_encoder">correction</corr> </choice>

Here is an example:

This would be tagged as follows:

<!-- ... --><l>Then rest thee in our porter’s keep,</l> <l>While our brother Francis will <choice>   <sic>re<rhyme type="dvpp:rhymeMasculine"     label="s1">pa</rhyme>   </sic>   <corr resp="dvpp:prs_ed_sk">re<rhyme type="dvpp:rhymeMasculine"     label="s1">pair</rhyme>   </corr>  </choice> </l> <l>To the house of woe, and soothe the <choice>   <sic>sou</sic>   <corr resp="dvpp:prs_ed_sk">soul</corr>  </choice> </l> <l>Of the dying man with <rhyme type="dvpp:rhymeMasculine"   label="s1">prayer</rhyme> !”</l>

The tagging of such errors is done at the word level or above; in other words, the smallest unit of text to be tagged this way is a word. ‘repa’ is first tagged as normal, with its final syllable marked as a rhyme with the <rhyme> element. Next, the word is wrapped in a <sic> tag. The <sic> is wrapped in a <choice> element, and a <corr> element is added as a sibling of <sic>. The complete replacement for the contents of the <sic> is included in the <corr> element, and the resp is added to specify who is responsible for the correction.

12.1. Substantive formatting versus incidental formatting

The DVPP project aims to present a relatively consistent rendering of poetry across the whole collection, but of course individual periodicals have different idiosyncracies in the ways they choose to lay out and style their poems. Some aspects of layout can be viewed as crucial components of the poem itself: for example, when line-indents vary within a stanza, the reader’s experience of the stanza will be different, because relationships between lines or sets of lines are made explicit. We view this kind of styling as substantive, and we always show it in the default rendered view. Italics used for emphasis would be another example of a substantive feature.

Other types of styling are incidental (not substantive): examples include illustrated capitals, small-caps rendering of the poem's initial word, layout changes caused by intrusive or enclosed graphical features and figures, periodical-specific styling of headings or titles, periods at the end of poem titles, and so on. We encode such features so that we can render them in a diplomatic view of the poem, but we also need to be able to suppress them in the regular view. This means that we need to distinguish the CSS for the incidental features from the CSS for the substantive features. We do this by using <rendition> elements with a specific string '_incid_' in its xml:id. You will see examples of this below.

12.2. Connecting CSS with the text

Our current practice requires that all CSS be provided using <rendition> elements in the header of the poem file. In earlier phases of the project, we allowed CSS to be included directly on elements in the text using the style attribute, but this approach is now deprecated and we will be converting all our legacy encoding.

Imagine that there is a short run of text which is rendered in ‘small-caps’ on the periodical page. Small-caps is a font variant in which capital letters are rendered as normal capitals, but lower-case letters are replaced by smaller versions of capital letters. It is quite common for the first part of the first line of a poem to be rendered in small-caps during the Victorian period. In order to capture this, we need to know:

• What is the CSS code for this? (Answer: font-variant: small-caps;.)
• Is this incidental or substantive? (Answer: this is incidental; it is a feature of the house style of some periodicals, and does not really alter the reader's experience of the poem.)
• Where do we put the CSS code? (Answer: in a <rendition> element in the <teiHeader>, as we'll demonstrate below.)
• How do we connect the string of text to the small-caps style? (Answer: by tagging the text with a suitable element, and then pointing its rendition attribute at the xml:id of the <rendition> element in the header.)

This is how it works:

In the <teiHeader>/<encodingDesc>/<tagsDecl> element, you create a <rendition> element like this:

The <rendition> element must have an xml:id which is unique throughout the project. To create the xml:id value, take part of the value of the xml:id attribute on the root <TEI> element, and add an underscore, then a brief descriptive word or phrase. The part you need to take from the xml:id is the first bit: pom_xxxx. That's enough to ensure that the xml:id of your <rendition> is unique. In the case of incidental styles, the xml:id must include _incid_ (which specifies that it's incidental), followed by something usefully descriptive. For substantive styles, just use something descriptive, without _incid_.

Next, in the first line of the poem, we add a <hi> element wrapping the relevant text:

The rendition attribute points to the xml:id of the <rendition> element.

12.3. CSS for layout

Layout information is usually associated with block-level elements. These are elements which form a block on the page (in other words, they have a line-break before and after them). In our TEI encoding, the principal block-level elements are:

• <div>
• <head>
• <byline>
• <lg>
• <l>
• <p>

There are a number of features we may want to capture for any block-level element:

• width: width: 25em;
• margin-left: margin-left: 0.5em;
• margin-right: margin-right: 1em;
• margin-top: margin-top: 2em;
• margin-bottom: margin-left: 2.5em;
• text-align: text-align: center;

This example encodes a stanza which has a top margin of 1em, and is indented (margin-left) by 3em (the width of 3 "m" characters); it has four lines, and the second and fourth lines are indented a further 1em:

(Note that the requirement to repeat the line indent rendition attribute on every second line in the poem can be avoided; see below for more details.)

12.4. CSS for text styles

CSS for layout (margins, widths etc.) can only be applied to block-level elements, but CSS for describing text styles (italics, small-caps, etc.) can be applied at any level, including block-level elements. When you see text which is distinguished in some way, first see if you can apply a useful tag to describe it; then create a suitable <rendition> element in the header (if you don't already have one which describes the same features) and add the rendition attribute to that tag, to show how it appears. Imagine that the phrase ‘a soul’ is in italics in the original. You may decide that this is because the author wants to emphasize it, so you can use the TEI <emph> element to tag it:

Then, in the header, you can add this:

and then add the rendition attribute to point to it:

Sometimes you can use a convenient tag that is already in place:

Sometimes, however, it's not clear why a particular piece of text is formatted in a special way, so you need to use a generic tag to wrap it. Use the <hi> tag for this:

These are a few examples of CSS properties and values you might use to describe text formatting:

• font-style: italic;
• font-variant: small-caps; (used when all text is in capitals, but "capital" letters are bigger than "small" letters)
• font-size: 120%; (size is relative to the existing inherited font size; also you can use 1.2em for the same result)
• text-decoration: underline;
• font-weight: bold;

12.5. Avoiding repetitive CSS

Often you will find that the same block of CSS code is repeated throughout your poem. Perhaps the last line of the poem is a refrain, which is always in italics, and indented by 2 ems:

It is tiresome to copy and paste that code repeatedly in the file. There is a simpler solution.

Go to your <rendition> element, and add a selector attribute like this:

What this says is ‘apply this style to every <l> element which is the last <l> in its parent element.’ Now you can simply remove the rendition attributes from all those lines.

What’s new and difficult here is the language used in the selector attribute. This is the language of CSS Selectors, which you can learn about on any of the sites which teach basic CSS. It can be a bit tricky to master, but in most cases on this project, learning a few simple recipes will enable you to do a lot of effective CSS encoding right in the header of the document.

13.1. Upper case versus title case, periods and linebreaks

Our site styleguide calls for poem titles and bylines to be rendered in sentence case, but the original print poems are often all-upper-case. In addition, our site styles provide standard font sizes for poem headings and bylines, but the original size and peculiarities such as letter-spacing should also be captured for the diplomatic view. Finally, titles in the Victorian period were often terminated with a period, but we no longer do that, so we need to make it possible for a final period to appear in the diplomatic view, but not in the normalized view. Here is a worked example.

Imagine the title you see on the page looks something like this:

THE VISION.⚓

However, we need to present the title in the normalized view as:

The Vision⚓

First of all, we transcribe in sentence case, including the period:

Now we create a <rendition> element in the header and link to it:

This ensures that when the reader switches to the diplomatic view, the text will appear in upper-case, and the font size will be set to the base font size (1rem), which will match the poem text itself, and the letter-spacing feature will be accurately represented. That just leaves us with the problem of the final period. Here we need to create two <rendition> elements, one to hide and one to show the period:

The result of this will be that the period is hidden by default, but when the diplomatic view is selected, the incidental stylesheet will be switched on, and the second rendition will go into effect, showing the period.

We can take a similar approach to linebreaks appearing inside poem titles. In our normalized view, we usually present poem titles as a single line, but the original may include linebreaks we want to capture. We use the same strategy as before, but now we need display: block; rather than display: inline;:

Now the title will appear on one line in the normalized view, but the linebreak will appear as soon as the diplomatic view is selected.

13.2. Mixed-column layouts

Some periodicals present poems at the full page-width, while others place them in a normal column layout in which the whole poem is contained within one column or runs from column to column. Neither of these cases is problematic; we write our layout CSS on the basis that the column is the containing ‘page’, and although we record column breaks with the <cb> element, we don’t worry about rendering them.

However, there are problematic cases in which a single poem has some components which span multiple columns, but has others that are contained within columns. For example, a poem might have its title, author and epigraph arranged so they span two columns, while the stanzas themselves are in columns. This is problematic because we have decided that both the normalized and diplomatic views should present the poem as a single column; we need a wide page to render the title matter, but a narrower page to arrange the stanzas in a single column. This section explains how we handle this in the CSS.

15.1. Original notes

15.2. Editorial notes

You will sometimes need to add an editorial note to the text you are editing. Use the <note> element to do this, with the resp attribute to specify who is responsible for the note. Here is an example:

Use the resp attribute to point to your own entry in the project personography.

16.1. Link Groups

The ana attribute is the mechanism that categorizes the linkGrp (in other words, describes the relationship which links them all). This set of values is built into the schema from a taxonomy which is stored in the xml/taxonomies.xml file. Each category in the taxonomy includes a short caption (the n attribute) and a longer explanation (in the <catDesc> element). As an encoder working on linkGrps, you do not need to worry about this; just assign the appropriate value to the ana attribute when you create the <linkGrp>.

Researching this grouping information can be tricky, and is best accomplished by using as many useful resources as possible, including the facsimile viewer and search page on the latest Jenkins build of the DVPP site.

Note that there is one additional use of the <linkGrp> element, where it has ana=lkSubcollection. That usage is discussed in 18. Site information pages.

19.1. How to refresh the database dumps and statistics

The MySQL database which stores all the indexing information is called ‘vpn’. Dumps from this database, in both SQL (vpn_live_latest.sql) and XML (vpn_live_latest.sql) format, are regularly put into SVN so that we have lots of options for recovery, rollback and forensics if we need it. The database server is set up to dump the two files every night at midnight into my home1t account, and you can get them from hcmc.uvic.ca/people/martin/vpn_latest.((sql)|(xml)) (note that the filenames are slightly different from those in svn). You can do this refresh automatically by running this in the root of the svn checkout:

ant adminRemote⚓

This brings down fresh versions of the dbs, and also refreshes the statistics file, stats.sts, which is used to track our progress over the long term. The ant task will attempt to commit the changes back to the repository, but that may fail if you don't have SSH keys set up with the svn server, so you may need to commit manually at the end of the process.

19.2. Refreshing the XML

This process must be undertaken regularly as long as the MySQL database is being edited. It is quite fraught with danger and complicated, so don't try it on a Friday afternoon with only ten minutes to spare; choose a time when no-one will be editing anything (db or XML), and give yourself plenty of time to check the results and fix any oddities.

First, it's important to explain what this process does and why it exists. The DVPP project has two distinct canonical data repositories: one is the MySQL database, in which all metadata about all poems is collected and edited, and the other is an XML document collection, in which transcriptions and encoding of actual poems is stored. The eventual aim is to store all data in the XML files only; they can easily handle all the metadata alongside the transcription, and they're must easier to process. But while the indexing process is still continuing, the database must be maintained, and the data collected and corrected in it is the canonical source for metadata for all the poems. At the same time, the XML files are the canonical repository for the transcription and annotation data. So inside the same TEI file, there will be data which is maintained by editing that file, and data which is maintained externally in the MySQL database.

Periodically, therefore, we need to refresh the XML collection to update the data in it with the latest changes and additions from the MySQL database. This involves:

• Creating a new TEI file for each new poem which has been added to the index;
• Updating the metadata in the <teiHeader> of any poem whose metadata in the database has changed since the last time the process was run;
• Creating a new TEI file with a new name for any poem whose title has changed in the database, because the title is canonical db metadata, and the TEI XML file name is generated directly from the title. When a new file is created for a poem which already had a TEI file, any transcription in the old file must be incorporated into the new file, and the old file must be deleted from svn when the new one is added.
• Any TEI files previously created for poems which have now been deleted from the database must be identified and purged from the TEI collection.

As you can imagine, this is both complicated and risky, so it is undertaken in stages, with human oversight.

ant adminRemote⚓

ant -f buildTEI.xml -Dperiodical=chambers_series -Dyear=1840⚓

A file called svnMoveObsoleteFiles.sh has been created.
 Check its contents; if there are no oddities, 
make it executable and run it to move
 obsolete files out of the xml folder.
 A file called svnAddNewFiles.sh has been created.
 Check its contents.
 If there are no oddities, make it executable and run
 it to add new files to the svn repository.⚓

svn commit -m "TEI file collection refreshed from MySQL database data."⚓

19.3. How to OCR a year

Our transcription and encoding process is able to go much faster when we provide OCRs of the page-images for transcribers to take as the basis for their work. The process is set up to run for all poems in the collection for a specific year, in a single operation; so you can OCR all of (e.g.) 1870 in one go. After making sure that your local svn copy is up to date, just do this in the root of the svn repo:

ant -f buildOCR.xml -Dyear=1870⚓

supplying the required target year. You can also OCR a single periodical for a single year, like this:

ant -f buildOCR.xml -Dyear=1870 -Dperiodical=chambers_series⚓

which passes the exact name of the folder for that periodical inside the xml folder. This is what the process actually does:

• Start building a shell script which will do the actual work.
• Check the TEI file for each poem in the collection which dates from the target year.
• If the poem already has transcription or OCR, ignore it.
• If the poem hasn't yet been transcribed or OCRed, process it as follows.
• Analyze the XML file to get the paths to page-images for the poem.
• Add lines to the shell script to curl the images for that poem, then run tesseract to create OCR output.
• Run the shell script to do all the OCR (this takes a while).
• Insert the results of the OCR into copies of the poem XML files, as a comment.
• Validate all the transformed XML files.
• If they're all valid, copy them back over the originals.

If this completes successfully, there's nothing else you need to do except commit the changes back to the repository. It's safe to run this process on the same year over and over again, as new poems get added to the index and eventually find their way into the XML collection, because poems that have already been OCRed or encoded are ignored, and only new files will be processed. However, if you need to re-process old files (perhaps because better images have been captured, or the original OCR process was unsatisfactory for some reason), you can add the flag

-DoverwritePreviousOCR=yes⚓

when running the Ant task.