Open Data is a valuable resource meant for everyone. But let’s face it – using data is difficult, and using it correctly is even more difficult. Our goal is to empower all San Diego residents, from the most techy savvy to the least, to get and use the City’s data.
We want to help our users by removing the roadblocks we have run into ourselves as data users. The first roadblock might be knowing how to use this data portal effectively. In this tutorial, you’ll learn how the portal is organized, how to search for what you want, and how to access the data once you’ve found it.
The main feature of this portal is a catalog of City data. The catalog contains hundreds of downloadable files, most of which are in a tabular format that can be opened in spreadsheet software such as Microsoft Excel.
Each of these downloadable files - called resources - has rows and rows of records. A record could be the transaction that occurs when a single business pays the City Treasurer for its license, or a record could be an incident that the Police responded to. A payment transaction and a police incident are very different events, so each of the records for those events will have different details. For example, one will have the name of the business as a detail, and one will have a name of a street where a public safety incident occurred. These details fill out the columns in the downloadable files.
Files with many rows can start to get large and difficult to work with. For this reason, the same kinds of records might be divided into multiple files according to year. Or, we may create a secondary file that aggregates the individual records into counts or sums by day or month. Files with the same kinds of records are grouped into a collection, called a dataset. The list of datasets is what you’ll see if you browse data in the catalog, but know that any given dataset may have many resources.
What if you have a simple question and don’t want to bother downloading and wrestling with big tabular files?
This portal can be useful to you, too. Our blog has data stories where you can see charts and maps containing some of the data on this portal.
To organize our data, we follow the Catalog > Dataset > Resource model briefly described above. This model is the standard for government open data portals and is published by the World Wide Web Consortium.
The portal’s catalog is essentially a list of datasets.
A dataset is a bucket for multiple views and formats of a single set of data, as well as a place to provide data dictionaries. Each of those permutations and formats is a resource, and all resources are linked from the same dataset page.
Resources are the actual files, APIs or links that we are sharing. We only provide resources that are machine-readable and in an open format. Resource types include:
Organizing the data into resources contained in datasets allows us to offer the same data in different formats or at different granularities.
If you need help interpreting the field descriptions, each resource has a corresponding dictionary available for download or preview from within the dataset.
To view all the datasets currently available, navigate to the DataSD home page, and click on the Browse Data menu tab at the top of the page.
With the search bar on the homepage, users can search for data based on keywords in the dataset title or the name of the publishing City department. Simply click on the Find Data search field, and key in your desired search term.
If you have a topical keyword in mind, try using this to find a dataset by title. Below is an instructional video guide, in which we:
If you don’t have a topical keyword in mind but have a good idea which City department publishes the data, try searching for that department. Below is an instructional video guide, in which we:
Not everyone knows how to work with data, and we happen to like doing it. Data stories are how City employees share deeper explorations and analyses of open data. City employees work intimately with data to do their jobs and can offer important context.
To view a data story, visit the Blog or view the See Trends section on the homepage. Click or tap the title to see the full story.
For example, to learn more about the number of planned solar installations in the City, you would click on the SOLAR PERMITS ISSUED title.