There are some fantastic developments in visualising data. From tag clouds to infographics to heatmaps to geospatial mashups to sparklines, finding new ways to understand and present data is essential in extracting value from the ‘data deluge‘, and solving the small, medium and grand challenges of our time. This excites me enormously.

It is, however, the domain of people who are cleverer than I (or at least much more adept at programming and using databases and analytics tools).

One of my major areas of work is on understanding and improving the whole of sector ‘landscape’ of data/information management in the environment sector. I’ve worked for the last eight years on strategy, policy and projects to help connect the many different datasets and system in this domain. This is in order to enable better access to knowledge generated from research, and better decision making and improved environmental management (including biodiversity, biosecurity, water and climate) by government agencies such as MfE, DOC, MAF Biosecurity, ERMA and the AHB, by local government agencies, and by NGOs and community groups.

By sharing data, and providing ‘middleware’ (such as the NZ Organisms Register) to connect data across different agencies, people have increased opportunity to develop and/or use tools to enhance the quality of the decisions they’re making, and the cost effectiveness of the limited resources we have for environmental management.

I’ve had a particular focus on information systems for biodiversity (the conservation of native species and ecosystems), but am now doing more work relating to information systems supporting biosecurity (preventing pest incursions and eradicating/managing existing pests).

Recently the Terrestrial and Freshwater Biodiversity Information Systems Programme (TFBIS) asked me to help determine where the gaps were in the biodiversity information systems ‘landscape’. I had written the TFBIS strategy in 2006/2007, and since then a number of systems have been developed to provide access to and connect existing datasets. The strategy helped give direction to approval of funding grants for such systems, but didn’t give a way of monitoring the progressive development of an interconnected and federated ‘meta-system’ for biodiversity management, or to understand which major pieces of ‘middleware’ needed to be developed next.

So, I made a biodiversity data landscape diagram. This shows the primary datasets, the sources of aggregated primary data, national middleware, web services, models & data transformation tools, interpretive tools, and user interfaces (for discovery, access, and data entry).

The diagram is a work in progress, and is very likely missing some items. If you know of anything that should be there but isn’t, please let me know. At the moment it’s a PDF, with many of the items linked to their web sites. In the future I’d like to create a more interactive version, hooked to a proper metadata repository.

It’d also be neat to see this approach used for other things like biosecurity, water, climate etc.

See the key to the diagram for descriptions of the types of items and definitions for each of the ‘levels’.

Next week I’m facilitating the ‘Research Data Matters‘ workshop for The Ministry of Research, Science and Technology, National Library of New Zealand and the Royal Society of New Zealand. This is a one-day event to discuss issues surrounding the long-term management of publicly-funded research data.

I’ve been working on research data policy issues with MoRST for about seven years now and its exciting to see how far we’ve come in that time. One of my oft collaborators at MoRST last week asked me whether I’d seen any infographics that represented the ‘data deluge’, in particular the figures cited in the article by that name from the Joint Information Systems Committee (JISC) in the UK.

I’ve seen some excellent ones on the size of the Internet, and file storage volumes, but nothing of that nature, so I decided to make one. This uses physical objects to show the relative scale of moving from a megabyte up to an exabyte. Click the image for a larger version:

Apparently the current size of the Internet is estimated at 5 trillion terabytes, or 5 exabytes. I note the JISC article is from late 2004, so estimates on the total annual production of information may well have gone up by then.

For those particularly interested the actual sizes, they’re not precisely scaled by 1,000 each time, but are fairly close. Here are the numbers:

Length of a tiny ant 1.4 millimetres Height of a short person 1.4 metres Length of the Auckland Harbor Bridge 1,020 metres Length of New Zealand 1,600 kilometres Diameter of the Sun 1,390,000 km

This infographic is licensed by Julian Carver under the Creative Commons Attribution-ShareAlike 3.0 New Zealand License.

First eMail, then eCommerce, eBusiness & eProcurement, eGovernment, eDating, and now eResearch. Does simply putting an ‘e’ in front of an existing practice make it somehow sexier, and more now? I headed along to the Wellington eResearch Symposium last week to find out.

OK, that’s not true. I did go to the Wellington eResearch Symposium last week, but I already have deeply held views about eResearch and have been advocating the concept for six or seven years. I’m just pretending to be a journalist today, and that sounded like something a journalist would say.

To read the rest of my write up of the event, visit my guest post about it on Sciblogs, the site that brings together the best science bloggers in the NZ on one website.

In season 1, episode 8, of Buffy the Vampire Slayer “I, Robot, You Jane”, Giles, the librarian comments to Jenny Calendar, the computer science teacher that what he doesn’t like about computers is the smell.

“What do you mean, computers don’t smell”

she says. Giles replies

“Smell is the most powerful trigger to the memory there is. A certain flower or a whiff of smoke can bring up experiences long forgotten. Books smell… musty and rich. The knowledge gained from a computer is… it has no texture, no context. It’s there and then it’s gone. If it’s to last, then the getting of knowledge should be tangible. It should be, um… smelly.”

I first met Paul Reynolds of McGovern Online (or @littlehigh as he became known on Flickr, Twitter and other social networks), at the National Digital Forum conference in 2007. The NDF is a “a coalition of museums, archives, art galleries, libraries and government departments working together to enhance electronic access to New Zealand’s culture and heritage”, something which I learned was very dear to Paul’s heart.

I had seen Paul on TV once or twice before, and admired his insightful and engaging style. We bumped into each other once or twice a year at conferences, or walking along Lambton Quay. I regularly listened to podcasts of his ‘Virtual World’ discussions with Jim Mora on Radio New Zealand.

Many of us in the Internet, open government, and open data space spent much of our formative years in the digital world. Playing video games as kids and teenagers, hacking on early home computers, and reading cyberpunk novels. The digital world had colour, and sound, but it was garish, tinny, maybe even a bit sterile.

What I loved about Paul Reynolds was the way he brought texture and richness to the digital world. He had a unique way of connecting the beautiful, tactile, physical, and even musty nature of art galleries, museums, and libraries with the expression of knowledge in digital environments. He seemed to understand the innately human aspects of both, and bridge them in a way no one else could.

He understood the relationship between content, people, and place in the physical world, and effortlessly applied that understanding to technology, the web, and social media. He did so in a way that was wry, amusing, and both pragmatic and visionary. He explained new things in ways that were easy to understand, often simultaneously with the excitement of a 7 year old boy, and the wisdom of a 70 year old man.

Paul, with your beautiful lilting accent, your expansive mind, and your love for literature, art, culture and technology, you gave the digital world texture, smell and sound. You shall be missed.