I am reaching a real milestone in the project now. It is a really weird, but good feeling that after the last few weeks of slog things are finally now falling into place and I can sit back a little and enjoy the end of this first phase.
Progress to date:
- WBGT and UTCI models understood
- WBGT and UTCI coded (python and C#)
- WBGT and UTCI tested and accurate
- Database connection and simple data extraction methods coded in python and python-MySQLdb API
Broken down like that, it doesn’t seem like an awful lot. But everything has been tested, verified and is good to go.
- Wrap up WBGT, UTCI and db connector into a simple, clean package
- Bruno’s challenge: Compare iterative WBGT algorithm with approximate WBGT calculation
- Matthias’s challenge: Update the database with revised Td approximation (and subsequent recalculation of WBGT)
- Build database interface, with common methods to update the database (add new data, update existing data etc.)
- Python: build python modules to interface with the database and perform these tasks (effectively separating data analysis needs from data storage needs)
- MySQL: create stored procedures to do the above directly within the database
- Benchmark the above to find the fastest, easy to use, reliable runtime bounds etc.
- Redesign the database table to take advantage of MySQL’s hash indexing and clustered tables
- Re-run the benchmarking tests to assess performance effects
It might be a big ask but I am putting a timeframe on this, planning to have the above completed by the time we return to polytech after mid-term break (14th October 2013). I am hoping to also have the report started, and these first phases written up. If I can achieve this, then I will be happy that I have investigated the current state of the project thoroughly and will have about 1 month to look at the disadvantages / advantages of alternate data storage systems.