Wednesday, June 09, 2004
Reflective chapter diary entry the first
For my thesis/dissertation/project report/whatever you want to call it I must write a reflective chapter about what I learned from the project, what went wrong, how corporate culture or what have you affected the project, yada yada. The reflective chapter receives a lot of weight when they're marking these things.
The report guidelines mention it might be useful to keep a diary as the project progresses. In case this might be of interest to someone and since Blogger makes storing it here easy, these are my thoughts after my first day of work. The project isn't confidential but I may edit this later if it seems like I can add in more details about what I'm doing without coming too close to spelling out exactly who I'm working for, or violating any confidentiality agreements.
For the next 3 months I'll work on determining if, given a bunch of computing jobs to be processed each with an associated deadline and possibly a priority, it's feasible to
a) figure out if they can be processed in time to meet the deadlines given the computing resource capacity/budget constraints and
b) if not, tell the person submitting or scheduling the jobs how much more CPU/server time they ought to buy in order to be x% confident that deadlines will be met.
While you can estimate how long it will take to process a job, given that it has certain characteristics, that estimate can be off by up to a factor of 3. Which makes calculating the probability that you'll be done on time with your current computing budget (or not, and if not, what is needed) a bit tricky.
It looks like I'll need to make some simplifying assumptions and then, once the thesis is done, see if they can be improved on or someone might have developed better prediction methods for how long job processing will actually take. There is intermediate feedback on progress as jobs are being processed, so it seems like Bayesian updating might be useful.
I ate lunch with the members of my team and people who are working on related problems. It was educational.
Some were critiquing another researcher's proposed solution for a different resource allocation problem -- it's brilliant technically, but it doesn't allow customers any certainty in reserving computing time. Several other reasons it might be a bad idea were hashed out, but one guy spelled out why he wasn't pushing back harder against the proposal. The person who's come up with it is a senior technology guru-type, with a lot of influence. All sorts of people end up doing favours for him. The guy whose work might be affected by this and who is not arguing too strenuously said something like, "I see the train headed towards me, and I'm not going to try to stop it only to get smushed. No, I will wait until a few seconds before impact and hop gracefully out of the way."
They then moved on to arguing whether another project is too ambitious, and it condensed a lot of the arguments from Problem Structuring Methods and OR Methodology into a two-minute back and forth of "they keep adding more and more stuff into their mathematical program, by adding more information from the real world to the optimization" "I don't think modelling the whole world will work, there's no way some of that data will be available in real life [eg. BT is not going to tell a customer their entire network topology], the real world is messier, and it makes more sense to find satisficing solutions to smaller, incremental bits of the problem than to have a tool that will give you an optimal solution but can't actully work." About 10 seconds into it I thought, "Wait, this is familiar, didn't this argument about the limits of optimization get worn out in the 1980s?" but apparently not. They also mentioned the problem of unforeseen traffic spikes, like if someone mentions your site in their blog. Anyway, interesting research questions, and the company politics seem intriguing as well.
p.s. If you are someone close to me, say for instance my mother or father, and want to know what my new address and phone number are and how I am doing and such and such, please rest assured I will be emailing the details you desire pronto. Like, by Monday. I've booked a car so that Jason can help me move the rest of my stuff Sunday, I am settling in nicely, and life is good though I still have a wretched cough.
