This can be fixed by centering the projection on the city’s centroid, then zooming and offsetting appropriately. This is most common when working with city-level GeoJSON data, which will appear as a tiny speck on the screen if the projection is not adjusted. I used it once to convert a shapefile of Westeros into a format that can be displayed in D3 (the final map, if you’re curious, is here: Game Of Thrones Interactive Battles Map). It has both a web interface and CLI where you can upload files in a variety of formats, edit their layers and polygons, and re-export them as GeoJSON or TopoJSON. You can edit and convert map data using a tool called MapShaper. Map data doesn’t always come nicely formatted as GeoJSON or TopoJSON sometimes they will come as shapefiles, which cannot directly be displayed by D3. To optimize page load time, you should always merge datasets and do preprocessing offline ahead of time. The best option is always to shrink your dataset, but if that cannot be done, then adding a loading indicator/spinner can go a long way to making sure users don’t think your visualization is broken while it loads.įor visualizations using static datasets, loading and combining multiple datasets or doing preprocessing on the data in Javascript is unnecessary. Larger datasets take more time to load, and more display elements take longer to render and update.Īs a rule of thumb, loading datasets with more than 25mb will lead to a severe and noticeable performance hit. Generally, page load time is related to the size of the dataset, and how much data processing/manipulation the visualization needs to do. The drawback, as I understand it, is that you can’t adjust the projection like you can with GeoJSON, so it is best to set your desired projection using a tool like MapShaper before you convert it. This approach can lead to very drastic savings - the entire SF Land Use dataset can be represented by a 32mb TopoJSON, which is more than 75% less! Check out the link above for more details about the TopoJSON format.Ĭonverting is as easy as loading the file into MapShaper and re-exporting as TopoJSON. GeoJSON encodes each shape separately while TopoJSON encodes each shared line once, so converting a GeoJSON to a TopoJSON saves a lot of space when there are many shapes with shared edges. If you don’t need a high level of detail, it is possible to reduce the complexity of the shape data by removing vertices, or reducing the precision of the coordinates. Removing whitespace reduced the size of the SF Land Use dataset from 98 to 93mb. The former can be easily done by loading the JSON into Python, then saving it with separators that don’t have trailing spaces. For example, removing spaces after each colon or comma, or shortening the names of properties/features. JSON data is stored as text, and thus we can save on file size by cutting down the number of characters we use. Applying this method reduced the SF Land Use dataset from 137mb to 98mb. Oftentimes, the datasets you download will have a lot of properties that you don’t plan on displaying, which can be removed to save on size. ![]() For example, in the SF Land Use dataset, each parcel is a feature and contains properties like their age and zoning type. Properties are like columns in a traditional CSV, they can hold data. ![]() Reducing the amount of properties for each featureĪ feature refers to some object on the map (like a building) that has a shape and some properties. This is something we want to minimize, and thankfully there are several ways to deal with this. Since D3 visualizations are rendered by the browser, it forces the user to download a massive data file to view the visualization, which makes load times exceedingly long. For example, the GeoJSON containing land use information on each parcel of land in San Francisco is ~137mb. ![]() Large GeoJSON Filesĭepending on the level of detail, GeoJSON can get very large. I plan to make regular updates/corrections as my understanding improves. This post is a collection of common problems people will encounter while making maps using D3, and my advice for how to deal with them.
0 Comments
Leave a Reply. |