Archive for the ‘Software’ Category.

Results of the metadata survey

Keeping track of what is recorded in the course of fieldwork is critical, both for your own future work and for longterm archiving. Recordings of dynamic performance (audio or video) are easy to misplace or misidentify and very difficult to locate once you forget what a file was named and what you recorded on a particular day. We ran a survey about how people record their metadata from January 21st to April 25th, 2016 and had 142 responses (see also the earlier blog post here). There were two multiple choice questions each allowing selection of more than one checkbox and the entry of free text responses. I can send the full results of the survey on request. This information will help inform the development of new tools for metadata entry. The responses are summarised below.

Continue reading ‘Results of the metadata survey’ »

Reading HyperCard stacks in 2016

HyperCard (HC) was a brilliant program that came free with every Macintosh computer from 1987 and was in development until around 2004. It made it possible to create multimedia ‘stacks’ (of cards) and was very popular with linguists. For example, Peter Ladefoged produced an IPA HyperCard stack and SIL had a stacks for drawing syntactic trees or for exploring the history of Indo-European (see their listing here). Texas and FreeText created  by Mark Zimmerman allowed you to create quick indexes of very large text files (maybe even into the megabytes! Remember this is the early 1990s). I used FreeText when I wrote Audiamus, a corpus exploration tool that let me link text and media and then cite the text/media in my research.

My favourite HC linguistic application was J.Randolph Valentine’s Rook that presented a speaker telling an Ojibwe story (with audio), with interlinear text linked to a grammar sketch of the language. I adapted that model for a story in Warnman, told by Waka Taylor, and produced as part of a set of HC stacks called ‘Australia’s languages’ and released in 1994. Continue reading ‘Reading HyperCard stacks in 2016’ »

Toolbox to Elan

In the spirit of solving small frustrations I offer my weekend experience of getting Toolbox files into Elan. I have over a hundred texts in Nafsan, most of which are time-aligned and interlinearised. I am working with Stefan Schnell on adding GRAID annotation to some of these texts and the preferred way of doing this is in Elan, with the GRAID annotation at the morphemic-level. I tried importing Toolbox files using the Elan ‘Import’ menu, and had listed all field markers in Toolbox, together with their internal dependencies (which should then map to Elan’s relationship between tiers). These settings are stored in an external file. Unfortunately, the import failed several times, despite changing the settings slightly after each attempt. Continue reading ‘Toolbox to Elan’ »

Somali phone dictionary

On June 13th we Arnold Zable will launch a Somali-English Dictionary app for both Android and iOS platforms, using the successful Ma! Iwaidja dictionary model. Opening screen of the Somali dictionary appThis is the product of a collaboration between the School of Languages and Linguistics at the University of Melbourne and Burji Arts, a Melbourne-based Somali arts and cultural organisation. The app contains some 26,000 Somali words and English equivalents with audio for selected items, so users can hear words or phrases. This app will have the capability of accepting input from users who can contribute items and suggest alternate pronunciations.

Continue reading ‘Somali phone dictionary’ »

Fieldwork helper – ExSite9

ExSite9 is an open-source cross-platform tool for creating descriptions of files created during fieldwork. We have been working on the development of ExSite9 over the past year and it is now ready for download and use:

ExSite9 collects information about files from a directory on your laptop you have selected, and presents it to you onscreen for your annotation, as can be seen in the following screenshot. The top left window shows the filenames, and the righthand window shows metadata characteristics that can be clicked once a file or set of files is selected.The manual is here:

Researchers who undertake fieldwork, or capture research data away from their desks, can use ExSite9 to support the quick application of descriptive metadata to the digital data they capture. This also enables researchers to prepare a package of metadata and data for backup to a data repository or archive for safekeeping and further manipulation.

Scholars in the Humanities, Arts and Social Sciences (HASS) typically need to organise heterogeneous file-based information from a multitude of sources, including digital cameras, video and sound recording equipment, scanned documents, files from transcription and annotation software, spreadsheets and field notes.

The aim of this tool is to facilitate better management and documentation of research data close to the time it is created. An easy to use interface enables researchers to capture metadata that meets their research needs and matches the requirements for repository ingestion.

Continue reading ‘Fieldwork helper – ExSite9’ »

Crowd-sourcing and Language documentation: September LIP

Ruth Singer recaps some of the interesting points of the last week’s Linguistics in the Pub, an informal gathering of linguists and language activists that is held monthly in Melbourne

The most recent LIP included a demonstration of the Ma! Iwaidja phrasebook and dictionary app developed by the Minjilang Endangered Languages Publication project (publishing as Iwaidja Inyman). The app and its planned stage 2 development to include crowd-sourcing of linguistic data, were used as the starting point for a discussion of the possibilities of crowd-sourcing in language documentation. The discussion was led by Bruce Birch of Iwaidja Inyman. Software developer Matthew de Moiser also took part, from the company Pollen Interactive, which developed the software. There was also a good mix of staff and students from all over: the Centre for Research into Linguistic Diverstiy (CRLD), La Trobe; Monash, RNLD staff and also two participants all the way from NSW: Rob Mailhammer (UWS) and Tiger Webb (Macquarie/ABC radio).

Crowd-sourcing is a way of gathering content for websites. The most familiar example of crowd-sourcing to most readers will probably be Wikipedia. The content is provided by willing volunteers and in the case of Wikipedia, it is also moderated by a volunteer community. Now that speakers of endangered languages are becoming more connected to the internet, via mobile phones, laptops and other devices, the possibilities for using crowd-sourcing to gather language data are becoming apparent.

A summary of the potential benefits of crowd-sourcing methods for language documentation was given in the initial LIP announcement (taken from an abstract of Bruce Birch’s):

The opportunities for language documentation presented by smartphone apps are that they allow people to record, annotate and upload language data as well as metadata in the form of audio, video, images or text. The process allows users of the apps to take advantage of the spontaneous opportunities for data collection which frequently arise, but which are often missed in the context of traditional fieldwork tools and methods. Crowd-sourcing facilitates the involvement of large numbers of native speakers of all ages in the documentation process without the need for high levels of literacy and computer literacy.

Bruce introduced some of the challenges and benefits of crowd-sourcing through a discussion of stage 2 of the development of the Ma! Iwaidja app, which he is currently working on. The aim is to create an interface which minimises the need for text literacy and technological literacy as much as possible. The current app, now available in iTunes, provides a phrasebook and 850 word dictionary. This is searchable and there are sound files for each phrase and word. It is also possible to record new phrases and words and store them on your own device.

The aim for stage 2 is to enable users to upload new words and phrases and share them with one another. It is hoped that speakers will be able to record themselves speaking for a few minutes about the meaning and use of any new words; essential information for a new dictionary entry. After being uploaded to a server, the new data would be curated and available as updates to users. Note that this model is a little different to Wikipedia as there would be a committee (the current Iwaidja language team) who would make decisions about what new data to incorporate into the app. One of the challenges would be how to fund this ongoing work, curating the new uploaded data.

Everybody was very impressed with the app as it is currently available. And so of course those working on their own dictionaries were keen to find out whether they would be able to use the software to make apps with their own language data. The good news is that the data was imported from Toolbox format. At some stage it should be possible to either buy the software (and put the data in yourself), or pay to have an app for another language built using the existing architecture. Bruce mentioned that it should be possible to get a phrasebook for under $5000. He and Matthew de Moiser definitely seemed interested in producing similar apps for more languages. There is a Mawng phrasebook app in production already and plans to create apps for other languages of north-west Arnhem Land (Australia) as well. There is also interest from the Northern Territory interpreters centre to produce a phrasebook with 100 phrases in 17 languages.

It was clear from the discussion that there is a lot of interest in the potentials of crowd-sourcing technology and apps for fieldwork and language documentation in general, so no doubt we will be hearing more about them from Bruce and other’s working on apps in LIPs to come.
The iOS version of the Ma! Iwaidja app is now available for download from iTunes for free (android to follow):


Is Toolbox the linguistic equivalent of Nietzsche’s typewriter?

There is an aphorism (apparently derived from Maslow 1966) that goes “if all you have is a hammer, everything looks like a nail”. For some documentary linguists reliance on the Toolbox software program means that everything linguistic looks like an interlinear gloss.

Toolbox (developed originally in 1987 as Shoebox by the Summer Institute of Linguistics) is a widely used data management and analysis tool for field linguists. It is designed for researchers to take units of transcribed text (typically ssentences) and semi-automatically “gloss” them to create multi-tier interlinearised text broken into words, which are then broken into constituent morphemes with aligned annotations such as sentence translations, morphemic translations, part of speech designations, and so on (for further discussion of interlinear text models see Bow, Hughes and Bird 2003).

Because Toolbox is free, and widely recommended for use in language analysis (it is commonly taught in training courses, such as InField, or ELDP grantee training, for example), it has had a large and constraining impact on how documentary linguists think they should do their research. I would suggest that it is a tool that has had a narrowing effect, like Nietzsche’s typewriter, as described by Carr 2008:

Sometime in 1882, Friedrich Nietzsche bought a typewriter—a Malling-Hansen Writing Ball, to be precise. His vision was failing, and keeping his eyes focused on a page had become exhausting and painful, often bringing on crushing headaches. He had been forced to curtail his writing, and he feared that he would soon have to give it up. The typewriter rescued him, at least for a time. Once he had mastered touch-typing, he was able to write with his eyes closed, using only the tips of his fingers. Words could once again flow from his mind to the page.

But the machine had a subtler effect on his work. One of Nietzsche’s friends, a composer, noticed a change in the style of his writing. His already terse prose had become even tighter, more telegraphic. “Perhaps you will through this instrument even take to a new idiom,” the friend wrote in a letter, noting that, in his own work, his “‘thoughts’ in music and language often depend on the quality of pen and paper.”

“You are right,” Nietzsche replied, “our writing equipment takes part in the forming of our thoughts.” Under the sway of the machine, writes the German media scholar Friedrich A. Kittler , Nietzsche’s prose “changed from arguments to aphorisms, from thoughts to puns, from rhetoric to telegram style.”

I believe that how annotation is conceptualised in language documentation, and presented in reference works like Schultze-Bernd 2006, reflects the narrowing influence of software tools like Toolbox and the dominance of interlinear glossing as an analytical method.

An alternative, developed originally by David Nathan, that we recommend at SOAS for corpus creation, is summary or overview annotation:

An overview annotation can be considered as a kind of “roadmap” or index of a recording. It could consist of approximately time-aligned information about what is in the recording, who is participating, and other interesting phenomena. For example, you could write:

“from 1 to 3 mins Auntie Freda is singing the song called Fat frog; from 3-7 mins Harry Smith is telling a story about joining the army; from 7-10 mins there is some interesting use of applicative morphology; from 15-18 mins contains rude content that should not be used for teaching children”
This could be written as prose (as above) or, better, structured into a table.

If you are familiar with software such as Transcriber or ELAN, you can do an overview annotation by marking breaks in topics/speakers etc, and typing descriptive text into the segments between breaks. Another strategy is to simply type a number into the time-aligned segment and then create a table which links the numbers with the overview information categories.

Interlinearisation of the Toolbox type is very time consuming (see my blog post on how much time transcription and interlinear annotation takes) while overview annotation can be done rapidly and relatively richly for a whole corpus, rather than the magical 10% of it too frequently referred to in the literature on linguistic annotation. This means that potentially it is a good alternative to the restricted representations that have been affected, like Nietzsche’s typewriter, by the very tool that documenters have come to rely upon.


Bow, Cathy, Baden Hughes and Steven Bird. 2003. Towards a general model of interlinear text. EMELD paper. [available online at, accessed 2012-04-21]

Carr, Nicholas. 2008. Is Google making us stupid? What the internet is doing to our brains. Atlantic Magazine July/August 2008.

Maslow, Abraham. 1966. The Psychology of Science: A reconnaisance. New York: Harper Collins.

Schultze-Bernd, Eva. 2006. Linguistic annotation. In Jost Gippert, Nikolaus P. Himmelmann and Ulrike Mosel (eds.) Essentials of Language Documentation, 213-251. Berlin: Mouton de Gruyter.

Books, HTML, audio, images – falling out from fieldwork

I’ll be going to Vanuatu next month courtesy of Catriona Hyslop’s DoBeS project, to help build an installation of three computer-based interactive dictionaries (Vurës, Tamambo and South Efate) for the Museum there. We will have hyperlinked dictionaries with sound and images where possible. All of this will be HTML-based for low maintenance and to allow new dictionaries to be added to the set over time. This post is aimed at outlining the method used to get these various files into deliverable formats and follows on from an earlier one where I talked about using ITunes to get media back to the village.

Continue reading ‘Books, HTML, audio, images – falling out from fieldwork’ »

A new transcription system

Just over a year ago I wrote a blog post about some of the parameters involved in transcribing media files, and how long it takes to do various sorts of transcription, translation and annotation tasks. In the commentary on my post, the ELAN transcription software tool developed at the Max Planck Institute for Psycholinguistics at Nijmegen came in for some criticism. Thus Ariel Guttman wrote that it was: “highly non-user friendly and non-ergonomic, especially since using the software only through the keyboard is not so easy” and “the people at the MPI should start designing their software with user-friendliness in mind”. Stuart McGill agreed: “you’re spot on with your comments on ELAN and keyboard use” and “transcription in ELAN is simply slow(er than it could be), no matter how well you know the program”. Stuart had decided that Transcriber, despite not handling special characters, was a better tool for his needs.

Well, as a result of user consultation involving Mark Dingemanse, Jeremy Hammond, and Simeon Floyd, the programmers at MPI-Nijmegen have now released ELAN version 4.1 which has a new “Transcription Mode” that Mark and Jeremy describe in a blog post as “designed to increase the speed and efficiency of transcription work. The interface is keyboard-driven and minimizes U[ser] I[nterface] actions”. Further details about the new mode and how to set it up and use it can be found in the blog post. It will be interesting to hear user reactions to the new facility over coming months.

Now, if someone would do a user consultation about the metadata browser IMDI, also developed at MPI-Nijmegen …