blog post inspired by T2Op

T2Op has had a greater impact than simply the two day conference. A successful webinar has been held and it has helped to scope a PhD.

In addition, one of the delegates has been musing for some time on some of the subjects raised (and more). He has recently published a blog post available here:

 

Threat to openness: managing access to public archives

 

Review of conference by a delegate

This review was written by Siân Morgan, who attended the conference on a student bursary.

On a cold, late November Tuesday, twelve speakers, spread across two days of discussions presented and shared their experiences of what Threats to Openness in the Digital World meant to them. There is, of course, not enough space to present all the topics and discussions that took place but hopefully an overview of these discussion can be achieved.

I was very fortunate to have attended on a student space, having recently finished my MA in Archive Administration. Throughout my course and my research project, there was an understandable focus on the role of the digital world on making information more accessible, but as was commented on, this is where the risk of access becomes stronger.

Julie McLeod opened the conference with reference to the terrorist attacks in Paris and how access to information and sharing information throughout the EU was being looked at more and more. The balance of security and privacy is one that reaches the papers everyday and Julie went on to question if technology was prepared for the challenges this would present. Would this have a positive impact on openness or would a failure of the technology actually threaten the openness?

Michael Moss and Tim Gollins’s The Digital Context got the sessions underway with the issue of sensitivity of records. What came out of this packed talk was, as Tim Gollins reiterated on the focus on sensitivity review, it is the Why, What, to Whom, When and in What context a records was created. The technological advances cannot compete with a human ability which identify these sensitive records. When entire files are made available, access is greater but so in turn is the risk.

David Thomas followed and his talk moved into discussing how the task of cataloguing the propaganda methods of ISIS is to a certain extent being neglected. The discussion of political Vs technological, David was firmly in the political camp. With social media sites such as Facebook and Twitter deleting accounts of ISIS users, it brings this leads to the question of who is cataloguing their communication, and one could argue their history. Understanding the recruitment efforts and the appeal of ISIS is a way in which access to the digital world can expand its aims for transparency.

A risk that was also acknowledged was a temptation to close records because of a lack of time and recourses leads to a risk of sleepwalking into a regime of closed access and this in turn breads conspiracy theories.

And so to Wednesday, the day ahead offered another wide range of speakers which started with David Edros on the legal perspective of threats to openness in the digital world. David Wilcox followed with a review of the work being done at the TNA, the successes enjoyed but also the challenges experienced. He gave such an excellent summary of the problems of transferring born digital record from public records to the TNA. Through the presentation, David Wilcox acknowledged the complexity of reviewing digital files owing to their fractured layout rather than the traditional page by page process.

Looking outside the UK was next with Agnes Jonker from the Netherland who detailed the experiences of the Netherlands with regards to access in the digital world with a simple but effective diagram of the changes in records management.

Arthur Lucas brought the discussion to question what is of historical interest, and who dictates it? One could argue that this can be used to control records being preserved, do we really need to keep 3 copies of the same report? If these were political reports and were presented to MPs and included scribbles in the margins, Arthur Lucas argued in the interests of openness, Yes.

We can’t second guess what people will be interested in the future, the scribbles on the margins are relevant, but if they are not included in the preserved document, full transparency cannot be achieved.

Mary Daly gave a fast paced presentation and showed exactly what can happen if events are not recorded and preserved. This lack of understanding, or perhaps the worry of decision makers to record their actions is a basic principle of record keeping and yet remained an issue that needed addressing. What was clear from the responses and the discussions throughout the conference was that the Threat to Openness in the Digital World varied from political to technological and the complex digital to the basic principles of record keeping.

Thank you again to the committee of the Threats to Openness in the Digital World for allowing me to attend and write a short piece. I hope I was able to highlight the main points of discussion especially the main points being made by the speakers, I am only sorry I could not include everything

Review by ITaaU
This excellent review was written by Michelle Pauli of IT as a Utility who helped fund the Conference, including this blog. It is also available on their website at www.itutility.ac.uk

Further observations from the Threats to Openness in the Digital World conference, Newcastle 24-25 November, 2015

Extreme weather metaphors were flying around at Northumbria University in Newcastle this week where the Threats to Openness conference was looking at the future of the public record in the digital age. According to the speakers, we’re facing a digital deluge in the coming decade as a result of a perfect storm of technological, political, legal and economic changes.

The fate of government records were at the heart of the discussions due to two major developments that are hitting archivists simultaneously: over a 10 year transition period that began in 2013 and involves releasing two years of records each year, records now have to be transferred to the National Archive after 20 years rather than the previous 30, and, soon, those sets of records will also be the first to be predominately digital. So, in 2016/17, digital records from the early 1990s – not least those dealing with decisions about wars in Iraq and Afghanistan – are due to be processed.

IMAG0766

This presents challenges at all levels of the transfer process, as David Willcox, digital sensitivity review lead at the National Archives, outlined. Digital records bring increases in both volume and complexity.

“We do not have dockets and files any more. We have blizzards of emails stored on different computers – a morass of stuff. The best thing you can say about it is that it’s data,” explained Tim Gollins, of the National Records of Scotland.

Two thirds of government data is held on shared drives, confirmed David Willcox. Email accounts for 50-70% of content, with one government department revealing that it had an impressive 190TB of email data.

This makes appraisal – the appreciation of the value and historical relevance of a record – more difficult. Even more challenging is sensitivity review – the process by which records are checked for compliance with data protection laws and any risks to national security, damage to international or business relations, personal information and so on. This review determines whether a record is retained by the department, sent to the National Archives as a “closed” record or opened to the public. According to Arthur Lucas, member of the Lord Chancellor’s Advisory Council, 75% of the documents going into the National Archives are closed, although he pointed out that transferring a document closed is not a way to “bury” a record and it is preferable to it being retained as at least it is indexed and its status can change in the future. Sensitivity review is currently done by humans, page by page, and can be a lengthy process. And it presents a real conundrum in the digital age: the nature of sensitivity review is inherently tricky for computers but sheer volume means that bringing in technology may be the only option.

Technology can certainly assist: eDiscovery tools can be used to apply categorisation or clustering to unstructured information; software can help highlight themes, events and people (which may help with reducing duplication – there is around 40% duplication in government records); 75% of exemptions from leading government departments relate to personal information and this is a good starting point for technological solutions as it should be easier for software to highlight easily identifiable fields such as names and addresses.

Research into this is crucial, and it is underway. Michael Moss, professor of mathematics and information science at Northumbria, and Tim Gollins have been working on an IT as a Utility Network-funded project to look at methods and algorithms that will enable the creation of useful tools. The CIA is working with the University of Texas on tools for CIA records. There is interesting work ongoing at Colombia University (they are “approaching record keeping from a completely different perspective,” commented Moss. “They have reconceptualised the archive from ‘a whole collection of documents’ to ‘data you can analyse’.”)

Why does this matter? “Transparency and openness,” said Willcox. “Good governance,” said Sir Alex Allen, former permanent secretary at the Ministry of Justice and Cabinet Office and author of the Records Review report into the readiness of government to move from the 30 year to 20 year rule. “Good record management is not just about preserving the historical record. It’s also important for the efficient running of the office.” When civil servants are asked for advice on a particular policy issue and they know that they looked at that same issue a few years ago, they need to be able to find the papers that relate to the discussions and decisions. It is also important for audit and accountability – how do you know if a private sector company contracted to do public sector work is fulfilling its contract if you can’t find the paperwork? – and for provision of evidence to public enquiries and legal proceedings.

A cautionary tale of what happens when it goes wrong was provided by Mary Daly, president of the Royal Irish Academy. She related the woeful story of how a key government decision was made during the Irish banking crisis in 2008. Or, rather, how we don’t know how that key decision was made because the records relating to it are “inadequate. In fact, non-existent. There was a complete lack of proper procedures.”

But the public record – and the huge changes it is undergoing in the digital age – is not restricted to government. According to Jeremy Frey, professor of physical chemistry at the University of Southampton, the scientific record, which is – or should be – also a public record when it is publicly funded, is also evolving.

“We are in a liminal period. Publication is a ritual we all go through but what we’re really about now is moving from paper publication paradigm to a digital one that will allow much more.”

However, there is currently a gap between the opportunities offered by digital and, in many cases, how they are being exploited – or not.

“Scientific papers have become a repository for the argument but most of the data is missing. In the past absolutely everything was in the paper, because it could be. Scale was not a problem. That has changed,” argued Frey. If researchers do not see and value the data then they cannot be sure that they can trust that value chain…and lack of trust in the data can destroy the scientific endeavour. As well as more intelligently accessible data (a scanned copy of data in the form of an image cannot be usefully searched) there needs to be more detail on methods and, to be disseminated effectively, it needs to have a narrative: “the story that you weave around the data that is as important as the data itself.”

For the humanities and social sciences, there is a challenge rearing from a different direction.David Erdos, lecturer in law and open society at the University of Cambridge, gave a rich rundown of the current state of the EU data protection landscape, how it is set to change and why it should concern humanities and social science scholars.

Currently, derogations within the directive allow some wriggle room for particular, special purposes, notably journalism, literature and the arts. The EU is now proposing that these are contained in a “middle area” covering knowledge facilitation more generally. One area of concern for researchers is that there would be no derogation from the proactive duty to provide privacy notices if the purpose of the data use changes. While biomedical research organisations have been busy lobbying about this, Erdos said that such activism needs to extend to the social sciences and humanities research community. “I have been trying to make them aware of it and their obligations around data protection,” he said. “The whole landscape is very confused around this and research ethics and policies. Seemingly, there is very little understanding of the implications of legislation. The community needs to fight for this – that’s what the press and journalists do.” It is likely that research will end up being an area for which a huge amount of discretion is passed to member states and, so, working on a national level will be as important as the European level.

Agnes Jonker, senior lecturer in Archives at the Archiefschool, (the Netherlands Institute for Archival Education and Research), University of Amsterdam, gave an insight into how the Dutch treat access to the public record, and how it compares to the UK. Most notably, the Netherlands’ first FOI law was drafted in 1980 (in the UK the FOI act is from 2005) and so there is, generally, a more relaxed air around the concept. Which is not to say that it is without its critics – a new FOI law was proposed in 2013 (though it will not be in force for a few years) to update legislation in the light of changes to the state: with the shrinking of government through increased privatisation, third parties are escaping FOI scrutiny. However, there is no reference in the proposed new law about the duty to document.

Coming a full circle back to public records and the humanities, Andrew Hoskins is a military historian who is concerned that current developments will render uncertain the record of warfare. He is particularly worried about the reduction from the 30 year rule to 20 years will result in more records being closed. “The buffer protects those potentially subject to embarrassment or danger. But buffering time is under pressure. From 2013 to 2023, two years of records will be processed every year without a doubling of resources. It’s punching holes in the records in demand by historians. It’s not a recipe for careful selection and preservation,” he said.

Hoskins, like Frey, thinks that the “story” is crucial – and risks being lost with the move to digital files. “It shifts away the context that comes from the handling of the physical file. Without the material context, the front and behind of each file, the information might be found but the story might be lost,” he said. “A history of warfare that depends on the official records of the British army has an uncertain future. Over the 2000s we’ve seen a perfect storm of technological, economic and political change at all points from collection to collation and archiving and assessment for declassification through to their being made public by the archives – and some of these pressures result from the culture of openness that has attached itself to current technological changes without adequate resources and understanding of the issues. It’s not a recipe for improved public access. Faster history is not necessarily better history,” he warned.

What’s the answer? The conference concluded by considering potential ways of moving forward and the actions, partnerships and collaborations required for that to happen. In the words ofDavid Thomas of Northumbria University, the ideal is to “take the archival idea and reinvent it in a new context of record making and record keeping in a new social world.” Watch this space.

 

Thank you and the future of this blog….

Thank you to all of the speakers and delegates who attended the conference, to our sponsors and to those who have tweeted over the last two days. It was a really interesting conference and has generated considerable discussion, musings and even some blue sky thinking. We enjoyed meeting you all.

So where do we go now?

This blog will continue to be updated. We will add slides and transcripts of some of the talks, and some of the photographs which have been taken. We will also be posting reports from several delegates, giving a variety of views and perspectives.

Of course there will also be data analysis of the the evaluation forms and the forms (and post its!) completed in the last session.

So keep an eye on the blog and twitter too.

 

 

Abstract of presentation to be given by Dr. David Thomas

Do changes to records technology or management practices threaten openness?

Dr. David Thomas, session 1

This paper is concerned with the question of whether the rapid changes to the ways in which information is recorded and managed in the digital world poses a threat to openness.  There is precious little point in worrying about FoI or Data Protection or anything else if records of value to researchers are not selected, don’t survive, or can’t be found.  I will also be suggesting that, in order to deal with some of the issues I have described we need to adopt a new and radical approach to thinking about digital records.  I’m going to end with a few paragraphs on a recent research project into open-source documents produced by  Islamic State which has some lessons for today.

 

Abstract of presentation to be given by Michael Moss and Tim Gollins

The Future Digital Archive and Sensitivity Review

Professor Michael Moss and Dr.Tim Gollins,  Session 1

We will look at the four elements in the archival process, appraisal, sensitivity review, curation and access. We will argue that analogue practice will not transfer seamlessly to the digital world, for several reasons – information is now produced on an industrial scale, it is rarely stored in any semblance of order, demand from users is changing radically and there are emerging computational tools that make the analysis of bulk data simple. Michael will provide an introduction and highlight why the challenge of digital sensitivity review is the most immediate of the threats to openness in the transition from analogue to digital. Tim will describe the overall challenges in digital sensitivity review and highlight initiatives and approaches that may begin offer a long term solution to one of the biggest obstacles to release of contemporary data into the public domain.