Monday, July 23, 2012

BOMs new data set, ACORN, so bad it's funny

What would YOU make of a data set that showed minimum temperatures larger than the maximum temperatures?  BOM is the Australian Bureau of Meteorology, Australia's authority on global warming (Don't laugh!)

When independent auditors found errors, gaps and deep questions about the HQ (High Quality) dataset for the official record of Australian temperatures, the BOM responded by producing a completely new set called ACORN in March 2012. But this set is also plagued with errors. One of the independent auditors, Ed Thurstan writes to me to explain that though the BOM says it aimed for the “best possible data set” and specified that they check internal consistency of data (one such check is to make sure that the maximum on any given day is larger than the minimum) when Thurstan double checked ACORN he found nearly 1000 instances where the max temperatures were lower than the minimums recorded the same day.
This raises serious questions about the quality control of the Australian data that are so serious, Thurstan asks whether the whole set should be withdrawn.
Why are basic checks like these left to unpaid volunteers, while Australian citizens pay $10 billion a year to reduce a warming trend recorded in a data set so poor that it’s not possible to draw any conclusions about the real current trend we are supposedly so concerned about.

The BOM goes to great lengths to assure us it’s high quality, peer reviewed, and rigorously checked, but with a days work, independent audits find major flaws

Errors in ACORN_SAT Data
Ever since the documentation for ACORN-SAT was released, I have had doubts about the ability of the Australian Bureau of Meteorology to honour their published intention to release all software that generated the ACORN-SAT data. ( I might amplify that thought later.)
In March 2012 the BOM released the report
“Techniques involved in developing the Australian Climate Observations Reference Network – Surface Air Temperature (ACORN-SAT) dataset CAWCR Technical Report No. 049  Blair Trewin
This specifies in great detail both the background to the development of the database, and the checks applied to the data. As Blair Trewin writes in the Abstract of this report:
“The purpose of this data set is to provide the best possible data set to underlie analyses of variability and change of temperature in Australia, including both analyses of annual and seasonal mean temperatures, and of extremes of temperature and other information derived from daily temperatures.”
I decided to take that document as a Program Specification, and write code to perform those data checks.
The very first check specified in section 6.1 of the above report is
“1.   Internal consistency of daily maximum and minimum temperature
 Since the temperature recorded at the time of observation (09:00 under current practice) is an upper bound for minimum temperature on both the day of observation and the following day (i.e. Tnd ≤ T0900,d and Tnd+1 ≤ T0900,d), and a lower bound for maximum temperature on both the day of observation and the preceding day (i.e. Txd ≥ T0900,d and Txd-1 ≥ T0900,d), daily maximum and minimum temperatures must satisfy the relationships:
Txd ≥ Tnd
Txd ≥ Tnd+1
If one or both of these relationships was violated, both maximum and minimum temperatures were flagged as suspect unless there was strong evidence that any error was confined to one of the two observations.”
In testing my code for the first of the two conditions specified above (which says simply that the maximum temperature recorded on any day must be greater than the minimum temperature recorded for that day), I found violations of this condition in the BOM data.
The following are extracts from the full violation log. The errors occur in many different sites and are spread across many decades:

In total, the ACORN-SAT database released in March displays about 1,000 (one thousand) violations of that simple rule that for any day

The Maximum Temperature must be greater than the Minimum Temperature.
This is a blindingly obvious type of error which should not have escaped quality control. It throws serious doubt on the whole ACORN-SAT project. In my opinion, these violations indicate that the entire ACORN-SAT database is suspect, and should be withdrawn for further testing.
Ed Thurstan
thurstan AT
July 16, 2012


Ambo wait puts lives at risk at Cairns

CAIRNS paramedics are being forced to wait more than one hour before patients find a bed at Cairns Base Hospital, a situation a key union says is putting lives at risk.

The waiting times, which can vary anywhere between 45 minutes to several hours, coincides with a spike in emergency calls in the past month.

As a result, paramedics and emergency department staff are being forced to engage in the practice of "ramping" patients on ambulance stretchers more often until hospital beds are made available.

The Cairns Post witnessed seven ambulances ramped outside the ED at Cairns Base Hospital’s ambulance bay yesterday afternoon, with eight patients waiting for beds.

The situation left only one remaining ambulance for the entire Cairns region.

When this unit was called to an emergency at Mt Sheridan, Mareeba ambulance station was the closest first responder.  [Mareeba is hours away]

"It has been a huge issue for us, probably our No.1 issue, for the last couple of years," United Voice Far Northern region state councillor Craig Crawford said.  "In the last month, it’s got worse, as in there’s been more patients ramped for longer periods of time."

The total ramping time for ambulances yesterday at the ED was 8.25 hours, with the longest individual ramping time for a patient clocked at 45 minutes.

Mr Crawford said while a range of factors was to blame for ramping, including a greater influx of hospital patients over the past month, paramedics were being frustrated by the "bed lock" problem.

"It’s heart wrenching for our crews because they’re standing on the ramp at our hospital listening to our comms department trying to get a crew … and there’s no one that can do it. It’s madness," he said.  "Someone could die out there in a house or in a car accident because we cannot send an ambulance, and we’re getting closer and closer to that mark."

Cairns and Hinterland Health Service District chief operating officer Robin Moore agreed ramping had been a strain on ED staff and paramedics, but denied there was a communication breakdown between first responders and hospital management.

"We have experienced unprecedented numbers through the ED, especially over this weekend, it’s been very busy," Mr Moore said.

"QAS and I are regularly in touch as soon as there’s any look of ramping, we’re in communication so QAS know what our processes are and how long they will be in that situation."

Meanwhile, Mr Crawford implored the government "to take control" of the ramping issue and implement the recommendations outlined in the recent Metropolitan Emergency Department Access Initiative to improve hospital management practices.


Bungling bureaucrats ready to flood Brisbane again?

THE State Government should speed up plans to reduce Wivenhoe Dam to 75 per cent capacity because of the threat posed by wet winter weather, an Ipswich councillor says.

Flood victim Paul Tully, whose Goodna family home was destroyed by last year's floods, is concerned that Wivenhoe has reached 99.4 per cent capacity.

"We have had a wet autumn and winter and there are only 19 weeks to summer but the dam is being held at levels reminiscent of the weeks leading up to last year's flood," he said.

"Wivenhoe Dam engineers admitted to the floods inquiry they paid no regard to weather forecasts in managing the dam and let nature take its course."

Weather bureau statistics support Cr Tully's concern about the wet winter.

Up to 60 per cent of the state experienced above-average falls in June.  Across the state, three daily rainfall records and four monthly records were broken last month.  The average rainfall was 33mm, which represented the highest figure in five years.

The wettest month was recorded by Tomewin, near Tweed Heads on the border of Queensland and NSW, where 338.4mm fell.  But places such as Noosaville had 293mm and the Sunshine Coast airport recorded 114.4mm in one day.

Seqwater confirmed they had begun minor releases of water from Wivenhoe after two weeks of rain.

But Cr Tully doesn't believe the state should wait until November to follow flood inquiry recommendations to lower dam levels to 75 per cent to accommodate the inflows from the coming wet season.

"It is incomprehensible they have buried their heads in the water and crossed their fingers hoping for a dry summer," Cr Tully said.

"Residents are fearful of another Brisbane River flood and feel the bureaucrats have abandoned them in their quest to maximise dam levels."

Cr Tully also suggested the State Government consider building another dam in southeast Queensland to enable Wivenhoe Dam to be used principally for flood mitigation.

"(Former premier) Joh Bjelke-Petersen promised Wivenhoe Dam would prevent major Brisbane River floods," Cr Tully said.  "Seqwater is playing with people's lives and again is mismanaging the operation of Wivenhoe Dam."


Why W.A.  teachers exit 'toxic' system

SWAMPED teachers who quit the classroom say their passion has been "killed off" and they feel "overwhelmed and undervalued".  And the profession has been described as "toxic" with a possible "crisis" looming.

The damning descriptions are part of exit surveys of 261 teachers and staff who resigned from the Education Department between January 2011 and January 2012, outlined in a report obtained by The Sunday Times this week.

The report also shows:

* More teachers blame poor work-life balance and workload pressures for their decision to quit, with those reasons cited in 13.4 per cent of resignations.

* Eleven per cent cited family reasons and just under one in 10 said they wanted to pursue other interests.

* Health issues were blamed by 8 per cent of those who quit in the past year.

* Staff said the department's methods for dealing with disruptive students needed the greatest attention.

* Seven out of 10 teachers leaving the department said they would consider returning in the future, indicating most were generally happy.

One experienced teacher, who described the first 10 years of teaching as "very rewarding", said the passion had been "killed off".

"Over the past four years, I have seen a steady decline by the department on the importance of children's self-esteem and social and emotional wellbeing only to be replaced by ridiculous tests (NAPLAN)," the teacher wrote.

"The children at my school need help to develop as a whole child as they have many home- life issues.

"I believe the department is only interested in achieving results and running schools like a business rather than thinking about what is important the children.

"I want my children to have fun while learning, develop skills and be happy. Not to just be able to spew out useless information so that their school 'looks good'."

Another employee called for a "closer look at schools with extreme behaviour management problems" because the administration was "encouraged to hide/cover up the behaviour management issues in their school to look better".

"If students are swearing, bullying and attacking other students and staff, they should have consequences," the employee wrote.

A qualified teacher told the department to "get your act together" after being told to "reapply for the same job every year, offering no job security" despite forecasts of teacher shortages in the coming years.

Another leaving employee said the department "should hang its head in shame" for the way it managed its restructure, from 14 districts to 75 school networks in eight regions.

"The last six months in this department have been terrible," they wrote.  "It is such a toxic work environment in here that it has affected me physically and emotionally. It upsets me greatly to sit back and watch such hard-working, dedicated officers treated so appallingly.

Another employee said "after many years of teaching", they felt workload pressures had become a "forgotten issue".

"Morale in schools is very low and as the baby boomers retire I fear a crisis will happen as younger teachers will not put up with the demands that are increasingly placed on teachers' time and efforts," they wrote.  "Much of the goodwill that teachers used to have has gone as they feel overwhelmed and undervalued."

Another teacher said they had simply "run out of steam to keep going" when they saw the "debacle with the national curriculum".

Some said they wanted more job security, better pay and greater recognition, while others said they loved their job and seeing the children flourish.

"My days at work were very happy and rewarding ones," one wrote. "I felt true achievement when in the classroom teaching students," another said.

Education Department workforce executive director Cliff Gillam said "only a tiny proportion" of leaving staff completed the voluntary survey, with 978 resignations, including 366 retirements.


1 comment:

Paul said...

Only one hour? I think you'll find its worse than that.

I could go on about the schizophrenic junkies that migrate up from the South for Winter because you can sleep rough without freezing to death, or the Aboriginals that come down from communities with the flying doctor claiming "chest pain" then use the hospital as a B&B while they bugger off for the day to visit cuzzinbrudders, or we could talk about Granny dumping because there still aren't anywhere near the Nursing Home beds needed, or the Nursing Homes that send in patients with sniffles only to see the family give the bed up because they don't want to see the inheritance spent....or we could talk about the New Guineans that I've mentioned before....but we aren't allowed, its officially all about a bad flu season.