NSW Police tackles ballooning data with dropbox



news The New South Wales Police Force has revealed plans to deploy a low level document management system somewhat akin to the easy access storage solutions offered by vendors like Dropbox and Box, as its existing systems continually struggle to deal with massively growing data volumes of files being used by its staff.

In a request for tender document issued this month and first reported by iTNews, the force noted that its 20,000 employees were currently producing “prodigious volumes of data”, increasing exponentially.

Currently, the organisation has total storage capacity of around 2500 terabytes, with about 887 terabytes being used for file content storage (rather than application databases). Most of this storage is used by evidence management systems such as the VIEW and Forensic Digital Imagery Management System (DIMS), which caters for visual imagery, CCTV footage and other sources of video, still photographs and scanned documents used as evidence.

The force’s request for tender documents state: “VIEW IMS and DIMS maintain strict chain of evidence capabilities. Once loaded into the VIEW system, files become managed according to investigation security controls inherited form corporate systems such as COPS and e@gle.i. Because of this, it is not broadly used as a general repository prior to analysis of the material (e.g. as a general “drop box” from mobile devices, from confiscated devices, or investigation materials).”

Neither, the NSW Police Force noted, could the e@gle.i system, which is the major investigation system, easily manage large amounts of electronic material, having limited capability in relation to video.

As a result, police officers were primarily using corporate shared drives as the holding area for files before they are uploaded into corporate systems, for files not required to be held in those corporate systems. The common shared drives are housed in a Network Attached Storage device along with several file servers also acting as storage devices. Currently, the tender documents note, there is no broadly accepted data management approach for digital content stored in shared repositories.

These files are considered more as the personal property of individual police officers, and are stored in such a way that they are primarily intended to be accessed by those individuals. This structure has created problems for the NSW Police Force — for example, tender documents note: “In some cases, sensitive or high volumes of material may be held on portable drives managed by the investigation area. This presents a number of challenges as investigations must be created urgently or individuals move on and workgroups disbanded as investigations close.”

Another issue is that the NSW Police Force’s Lotus Notes email platform is also used extensively as a file exchange facility. The tender documents state:

“In many cases, it has become the default storage and retrieval data management system with the content of email used as retrieval “metadata”. This creates large amounts of duplication of files (especially as attachments) and introduces access control risk. The email system has recently introduced an email “vault” to assist in mitigating file duplication, quota restrictions and lost email issues; however, email remains one of the main data management tools available to NSWPF staff (beyond its main purpose of message exchange).”

“Unfortunately, this means added demand for space on the shared common storage repositories and email storage systems. As the capacity of these repositories is expanded to accommodate the growing need for space, search and access times become progressively slower. At the same time, the management and storage of data is becoming increasingly challenging and expensive. The indeterminate nature of storage needs also often makes capacity management and storage service delivery fall short of expectations.”

Lastly, the NSW Police Force is moving to greater use of” bring your own and provided mobile and digital recording devices in the field”.

The growing use of these devices, according to the organisation, will place even more pressure on its storage facilities. There is currently no common, managed device synchronisation or file “drop box” capability to support a diverse and growing number of devices including bring your own devices likely to be introduced in the near future.”

NSW Police’s requirements for a catch-all solution to all of these problems appear to match most of the functionality currently provided by a corporate ‘dropbox’ solution as is provided by US-headquartered corporate file-sharing platform Box, which offers content management and corporate administrative control on top of its platform. However, it is unclear whether an Australian law enforcement agency such as the NSW Police Force would be allowed to use a cloud-based solution which may fall under the requirements of the US Patriot Act and other US legislation.

Close observers of the NSW Police Force’s IT unit will no doubt agree with that that it is rapidly emerging as one of the more innovative public sector organisations out there at the moment, as it grapples with how to best use technology to make its policing operations more efficient and effective.

In September last year, the force revealed plans to undertake a four-week field trial of a mobile app for officers to issue traffic infringement notices, which will be deployed in the field on locked-down versions of Apple’s iPad mini tablets. In August 2012 it was a huge internal social networking platform to allow police officers and admin staff to better communicate. It’s even seeking to replace its venerable COPS policing platform. And now we’re seeing a Dropbox-style deployment tendered for to handle the massive flow of data coming into the organisation from police officers in the field.

There’s no doubt that not all of these projects will be delivered as planned. Policing is messy work; things go wrong, there are constant edge cases, and it’s overall a very complex and challenging environment. This is a massive organisation with very difficult requirements.

However, I want to applaud the NSW Police Force’s IT team for tackling those challenges head-on. This kind of storage deployment is precisely what we need to be seeing from police organisations — it’s flexible, catch-all, and tailored to officers’ needs. Let’s hope we see more of this from police organisations around Australia.


  1. I’m a little concerned: What type of data will they be storing in the cloud? Sounds like there will be a lot of sensitive data being uploaded to American servers… For easy access by the NSA and whatnot.

    If only we had an all-fibre national broadband network… so that Australian companies subject only to Australian law could provide domestic data storage here in Australia…

    Oops, that topic’s taboo.

    • There are plenty of ‘host-it-yourself’ alternatives to dropbox, like Owncloud or Sparkleshare, so there’s no need to assume that it’ll be uploaded to US servers.

      Of course, any private network you may build will probably run on comms gear with backdoors built in by at least one foreign government…

      • Oh, I misread. I thought it was going to be actually stored in Dropbox, rather than a Dropbox-style deployment.

        Hmm, questions questions…

Comments are closed.