Sunday, January 31, 2010

Update on new Software List

Well, I now have the "import" working on the new Software List, and I've cleaned up the code and moved common functions to a separate file. I'll leave any further changes until next weekend, but in the meantime I've released this to a few folks for testing. We're finally getting there!

Later, I'm going to add some fields to the database, to keep the name of the binary package (on ibiblio) and the corresponding source package. This won't show up in the Software List, and it will be optional to add them, but (if present) they'll get sent to FDUPDATE automagically.

Also later, I'll add some fields for the URL (on for the license, and an indicator for whether or not the FSF considers this a "Free" license. These won't be used by the Software List, so they won't normally be visible, but whoever puts together the next FreeDOS distribution can use these for reporting.

As part of the above, I intend to write a report utility to check for errors in the data, and otherwise show some useful statistics.

Some answers to questions you may already have:

Q: Why make the Software List into a database?

A: Using a database (with a web front end) means we can automatically make sure things get entered in a consistent way. This will make the Software List better for everyone. This also means we can export the data in any number of ways. For example, FDUPDATE can get a list of all programs in XML format. End users see the normal html version of the Software List. Developers can export an LSM version of program information.

Q: Who updates the data?

A: We can set up "Software List administrators" (for lack of a better term) who can import and edit data. These people do not (necessarily) need to be web site admins, or news admins. It's a different role.

Q: How will it get backed up?

A: I plan to create an automated job that will export all the data (probably weekly) into LSM format, and keep that copy on ibiblio as a backup of the data. If the worst happens, at least we'll still have the program data.

Q: What happens if we lose the data in the database?

A: If the database just gets deleted (or corrupted) then a "Software List admin" can import all the LSM files from ibiblio. It's a slow way to do it, but it works for disaster recovery. We can also make some sort of remote backup using mysqldump, so you can just re-import that file.

Monday, January 18, 2010

What I'm working on now

Back in February 2009, I had announced that I was taking an absence from the FreeDOS Project to focus on a Masters program in Management of Technology (MOT). Over the next few months, I transferred my roles in FreeDOS (webmaster, SourceForge admin, ibiblio admin, etc) and finally stepped aside in May 2009, with Pat Villani now acting as the new FreeDOS Project Coordinator.

Since then, some very positive opportunities have come my way, and I have decided to defer entering the MOT program—at least for now.

So what have I been doing, if I haven't been busy in MOT?

You've probably noticed that I've been making updates to the FreeDOS web site. Some changes are very visible (the web site redesign, for example) and others not so visible. One of those "behind the scenes" projects that I'm working on at the moment is building a new version of the FreeDOS Software List.

The current Software List is a set of simple perl CGIs that act on a set of regular LSM files, displaying them in various tables. It was an easy solution for how to quickly and easily display details about the programs we include in the FreeDOS distributions. To manage the Software List, a webmaster just uploads an LSM into a particular directory, and the CGIs do the rest.

I once wanted to write a set of CGIs that would allow "Software List administrators" (for lack of a better term) to update software list info via a web form, and keep all the data in a database, so not have to upload files anymore. The idea was that an "admin" should be able to edit fields directly, or upload an LSM and let the system parse the LSM appropriately before importing fields into the database. And be able to display the data via web pages (like we do now), or as XML (useful for FDUPDATE), or as a correctly-formatted LSM file (for the FreeDOS Install program.)

That's what I'm working on now. The "Software List" part is working great, and looks very nice. (Sorry, I don't plan to share the URL until this is closer to "live", so no one gets confused when they look at "test" data.) The "Editor" function seems to be solid. Right now, I'm trying to finish up the "Import" process.

Once that's done, I plan to do some code optimization, identify duplicate code and move those into functions, that sort of thing. Later, I'll write a "Report" function that generates a status of the Software List, useful in finding/fixing missing or bad data.