The big day is coming ... New Sources!

I've finished the preliminary data load from Lewis and Short's "A Latin Dictionary". For those interested, there are approximately 51500 lemmata* (compared to 17500 in the Lewis Elementary).

The data is not live for users yet, so you won't be able to see the new data. But I wanted to tell you about this great breakthrough that I've made. The data needs a little bit more massaging to be considered production ready, but it's very, very clean data so far. I spent several weeks mining the data, and my heuristic algorithms** are getting pretty smart. A few more tweaks!

The big news is how this is going to affect the dictionary. I plan on adding a smart option-box in the bottom right-hand corner that will do two things. 1) List which dictionaries are available for searching and 2) allow the user (that's you!) to change the order in which they are searched and turn them on or off. You can see my mockup of this concept to the right. (Incidentally, this is my first post with graphics!)

The biggest benefit of the LNS (Lewis and Short) dictionary is that it contains 3 times the number of words -- granted, most of them are proper names and place names, but sometimes it's nice to know who and where those crazy ancient authors are talking about.

Keep your eyes peeled. It will only be a matter of weeks until this new data is live!

*For those who don't have experience in the field of lexicography, a lemma is a "head word" ...
** Experience-based methods ...

Regular Updates

What's going on now? As always, work goes on in the background, but nothing big has changed. Regardless, I want to make sure the front page stays fresh, so this update is to let you know that I continue to make small improvements to the dictionary data and the paradigms that the Latin parsing engine runs on.

Mostly, I spend a good amount of time correcting bad data, fixing wonky definitions, etc. But from time to time I find an error in a paradigm (for instance, recently I found macrons on -unt verbs) and fix it. Just a few days ago I discovered that short form 3rd adverbs like potenter don't parse properly; that's one of my current projects.

What's going on in the future? I'm still pulling data out of the big Lewis dictionary. I haven't loaded it yet because I'm happily discovering that the big Lewis dictionary has a wealth of information that can be extracted. The hard part, as it turns out, is extracting it. I don't know when the new data will be online, but rest assured I'm working on it often.

Until next time, feedback is always welcome!

Regular Progress

I like to keep the news relatively fresh, even if nothing major is happening. But even though this is a somewhat slow time for Numen development, there are still a constant stream of small updates. I'd like to keep everybody apprised of the situation.

One of the benefits of having students who actively use this dictionary is their feedback. One of the things they noticed was that sometimes an error message pops up saying, "AJAX has timed out", and it would happen relatively often -- especially on the flashcard practice tool. So I dug into the code and found the issue: I designed the site so that -- if any request took more than a few hundred milliseconds -- it would give an error. Such timeouts always involve a balance between briefness and lengthiness of waiting, and I quickly realized that I had not struck that balance. So I bumped up the timeout duration to something reasonably middle-of-the-road and. Voila! Problem solved! Gratias vobis ago, discipuli.

More: My big project with Vergil and Livy is still paying off. I continue to correct dozens of tiny mistakes and errors in the data every week, and I was able to run some statistics. Excluding proper names and place names, the parsing engine can analyze and pin down around 98.5% of all the words in these two Augustan authors. So accuracy is definitely improving daily! I can't yet account for all false positives, but they seem to be less than a fraction of a percent (anecdotally).

Speaking of accuracy, there is still room for improvement in three key areas:

1. syncopated forms (laudaverunt => laudarunt), which Livy loves by the way!
   
2. irregular forms (bobus, filiabus)
3. proper and place names (which do not, for the most part, exist as a regular part of the Lewis Elementary dictionary).
   

I have been cogitating over solutions for all three issues, but it might take a while to implement them -- probably not until December or January (winter holidays, yay!).

In other news -- I guess I have more than I had first assumed -- I've almost got a word list feature finished. This is for people who prefer to work with formatted word lists as opposed to flashcard decks (which I understand are sometimes referred to as index cards). The only major problem I have with this process is that the Lewis Elementary dictionary does not provide a "core" definition for most words, so the word list would have extremely lengthy definitions. I think one of my best options is to import the data from Whittaker's Words, which have more simple, more core-like definitions. But this could be problematic as a 1:1 mapping between Lewis' forms and Whittaker's forms would be difficult to achieve. Alas, I shall continue to think on this one.

Okay, so that's enough for now! Keep using it, and please keep reporting problems and errors! It may take a few days or weeks, but I eventually do fix all the errors!