Hosted Open Data Portals
Let’s make Open Data easy for small governments, non-profits and communities
The Civic Knowledge project aims to make it much easier and less expensive for small governments, non profits and community data projects to publish data to the web. We’ll be bringing CKAN, the software that powers some of the largest open data sites in the world, to the smallest government, civic and social organizations as a hosted webservice.
Implementing an Open Data policy requires having a data repository, a publicly accessible website where data can be published. While there are some excellent options, both commercial and open source, they are still too expensive for smaller cities, counties and non profits to adopt. By making is easier and less expensive to publish data to the web, we can encourage more cities and counties to adopt Open Data policies.
This project will provide a hosted data repository that is free for most users, and much less expensive for others, with predictable pricing and excellent administration, with the goal of making publication of Open Data as easy as publishing a blog post. In addition to the usual target of Open Data efforts — state and local governments — this project will also consider the needs of other organizations with substantial stores of unpublished social data, such as non profits, community indicator projects and community data intermediaries.
Interested? We’re just getting started, so contact the project lead, Eric Busboom, at firstname.lastname@example.org or 858 386 4134.
Publishing Data Is Too Expensive
Unlike general web content management, where there are hundreds of blogging, social media and photo sharing applications and services, there are few options for publishing data to the web, with only a few dedicated applications, such as Socrata, CKAN and DKAN. The primary commercial offering, Socrata, which can cost a small city $50K per year, is too expensive for smaller cities, and the Open Source options, CKAN and DKAN, are best suited for organizations that have technical staff or can hire contractors. Cities, or departments with cities, that have small budgets and no IT staff have few options for easily publishing Open Data.
These problems are limiting the number of governments and non profits that are able to publish Open Data; Open Data policies are useless if the software doesn’t exist to publish data. Simple technical and cost issues are limiting the extent and adoption of open data practices and policies.
What is the right price? At its simplest, publishing Open Data can be nothing more than blog posts with links to spreadsheets. While that isn’t ideal, it does suggest that the most common price for data publication should be similar to the most common price for blogging: zero.
What Small Data Publishers Need
To encourage wider adoption of Open Data policies, cities, indicator projects and non profits need a way to publish data that is:
- Easy to acquire
- Easy to use
- Easy to integrate into existing websites
Satisfying these needs would have a significant impact on Open Data programs nationwide by increasing both the number of open data websites, and, with better user interfaces, by making it easier for people to find data on those websites.
With the Civic Knowledge project, we expect to extend the excellent foundation built by Open Source offerings like CKAN and DKAN to the smallest governments that have the least capability to manage additional IT projects.
What We Will Provide
This project proposes to extend existing open source data repository software ( CKAN ) to be able to operate as a large-scale hosted webservice that can provide open data repositories to non profits and small governments for free, and to larger users at a very low cost. This web service should be coupled with open data advocacy and consulting service, possibly in conjunction with an existing organization like the Sunlight Foundation or Code For America, to provide a simple way for governments and non profits to get help with the whole process from articulating a desire to publish open data though policy design, policy adoption and technical implementation.
This proposed project, which we will refer to as Civic Knowledge, would be a complete Open Data solution for small cities and counties and provide back-end data management for community indicator projects, with services that include:
- Advocating Open Data policies to governments and non profits
- Help crafting Open Data Policies.
- Offer software and service to support Open Data.
For light use, fewer than 250 datasets and 10,000 visitors per month, the service would be free. Premium services would include enterprise features, such as organizing users into departments, connections to enterprise directory services, and other support required to integrate into enterprise IT services. Civic Knowledge could also offer services for managing the data portals, such as:
- Collecting, cleaning and publishing data
- Documenting and categorizing datasets
- Data Librarian services, helping people find and use data
Who Is This Project For?
Civic Knowledge will be designed to be appropriate for the smallest organizations, even unincorporated citizen groups and small city departments. For most of these organizations, Civic Knowledge will be free, and for more sophisticated use, it will be very inexpensive, starting at less than $75/month, excluding storage costs.
- Small cities
- Departments of larger cities
- Non profit social organizations
- Community groups
These small governments and non-governmental organizations have a lot of data that they’d like to share, but they don’t, in part, because it is too much work to do it right. Let’s make it easy for them and make Open Data as pervasive as blogging
Interested? Contact the project lead, Eric Busboom, at email@example.com or 858 386 4134.
What is CKAN?
We’ll be building on CKAN, taking advantage of its many features, and add to it features to allow for running in a dense, clustered computing environment, and interface additions for smaller organizations that can’t afford a dedicated administrator.
In 2013, Data.gov switched from Socrata, a commercial data repository, to CKAN. Data.gov now hosts about 100,000 datasets for dozens of federal departments, and links to many states. Many of the largest departments, like the Department of Energy and Health and Human Services have their own sub-repositories.
What We’ll Do
This project will produce a set of Open Source software, to make it easier for organizations to host their own data repository, and run a hosted service for those that don’t want to manage servers.
The software products will include:
- Scripts for deploying CKAN
- Pre-configured virtual machines, for fast deployment to Amazon EC2 or VMWare
- Extensions to CKAN to make it easier to administrate
- Creating and deploying Open Data policies
- Curating and Managing data
- Anything else required to get cities publishing Open Data.
Can you Help?
If you’d like to support this project, we could use you help. The project is just getting started, so we are recruiting:
- Beta customers, both cities and non profits
- Financial support
- A project manager
To get started on a project that will drive the next stage of Open Data, contact me, Eric Busboom, , at firstname.lastname@example.org or 858 386 4134.