Week #10. Network analysis in GNM

The new GNM functionality came to GDAL with two new applications: gnmmanage and gnmanalyse. In this post I'll describe the new classes and methods of API by example of these apps, which I also implement on this week: this is my last commit.

Note: I've also renamed some classes (e.g. GNMConnectivity -> GNMNetwork) and moved some methods from one class to another, due to the better correspondence to my concept. I'll describe the final architecture appearance a bit later, but it hasn't been changed a lot.

gnmmanage

The gnmmanage utility intends to provide all managing actions with networks: creating/deleting networks, creating/deleting features, setting/removing connections. For now it has the following usage:

>gnmmanage.exe --long-usage

Usage: gnmmanage [--help][-q][-quiet][--utility_version][--long-usage]

[create [-f format_name] [-t_srs srs_name] [-dsco NAME=VALUE]... ]

[import src_dataset_name] [-l layer_name]

[autoconnect tolerance]

[remove]

[-nt]

gnm_name

create: create connectivity or the full network, depending on native format support (-nt)

-f format_name: output file format name, possible values are: [ESRI Shapefile]

-t_srs srs_name: spatial reference input

-dsco NAME=VALUE: dataset creation option (format specific) import src_dataset_name: dataset name to copy

-l layer_name: layer name in dataset (optional)

autoconnect tolerance: create topology automatically with the given double tolerance

remove: remove connectivity or network, depending on native format support (-nt)

-nt: use native network format (now unavailable)

So the way using utility is similar to my previous utility: gnminfo, which soon will be using only for getting info about the existing networks. All managing functionality will fully migrate to gnmmanage and it will be more close to the gdalmanage.

The new method GNMNetwork::AutoConnect() tries to build the topology of the network automatically, as I described in my previous post. Remember, that if you will try to build a topology on several layers two times in different spatial reference systems - the results will be different. For example in our case the common network's SRS is 'EPSG:4326'. If we create a network in 'EPSG:3857', import all layers as previous and AutoConnect them with the same tolerance again (for me it was 0.00005) - the topology will differ from previous case.

gnmanalyse

The gnmmanage utility intends to provide all analysing functionality of GNM. It has the following usage:

>gnmanalyse.exe --long-usage

Usage: gnmanalyse [--help][-q][-quiet][--utility_version][--long-usage]

[dijkstra start_gfid end_gfid [-ds ds_name][-f ds_format][-l layer_name]]

gnm_name

dijkstra start_gfid end_gfid: calculates the shortest path between two points

-ds ds_name: the name&path of the dataset to save the layer with shortest path. Not need to be existed dataset

-f ds_format: define this to set the fromat of newly created dataset

-l layer_name: the name of the resulting layer. If the layer exist already — it will be rewritten

For now it has the usage of only one method: GNMStdRoutingAnalyser::DijkstraShortestPath(). For example lets use it on the network, which had been built with the gnmmanage, the following way:

>gnmanalyse.exe dijkstra 718 1185 -ds ..\shortestpath.shp -f "ESRI Shapefile" -l shortestpath ..\network_data

Path between 718 and 1185 found and saved to the ..\shortestpath.shp successfully

So the path between features with 718 and 1185 GFIDs will be found and saved into the instantly created or opened Shapefile layer.

Week #9. Working with features and gnminfo python tests

There are several methods were added to gnm: CreateFeature(), SetFeature(), DeleteFeature() which do the required actions for the connectivity and then call the corresponding GDALDataset methods.

This week I've also finally coped with python in GDAL. So for today there is a working python test which tests the gnminfo utility and can be used in command line like the following (this was tested on Windows):

D:\GitHub\gdal\autotest\utilities>c:\python27\python.exe test_gnminfo.py
  TEST: test_gnminfo_1 ... success
  TEST: test_gnminfo_2 ... success
  TEST: test_gnminfo_3 ... success
  TEST: test_gnminfo_4 ... success
  TEST: test_gnminfo_5 ... success

Test Script: test_gnminfo
Succeeded: 5
Failed:    0 (0 blew exceptions)
Skipped:   0
Expected fail:0
Duration:  0.47s

Found libgdal we are running against : D:\GitHub\gdal-build\bin\gdal20.dll

, where test #1 is simple --help test, test #2 creates a connectivity in autotest/tmp, test #3 gets info about it, test #4 imports some layers from autotest/gnm/data and test #5 deletes the connectivity.

Week #8. Automatic graph building

According GitHub commit: https://github.com/MikhanGusev/gdal/commit/12dc0678c78329f65f885958103e056491fadcd1

Automatic graph building in GNM

Similar to my old concept there can be several automatic graph building algorithms in GNM, because this mechanism is implemented as the pattern "Strategy". Firstly we must initialize the strategy with its own settings, than set the strategy for the GNMConnectivity via SetAutoConnectStrategy() and finally call the GNMConnectivity::AutoConnect(). The system graph layer will be filled with new connections.
Now there is only one algorithm in GDAL source tree - the one, which I'd implemented already in an old API. Now it is integrated to the GNM and builds graph basing on the set of line and point layer ids of the current GNMConnectivity.
So now GNM has a capability not just to create a set of obligatory layers over a dataset, but to create a real connectivity among features in this dataset, if it is needed.

Features

* We can modify any built connection via GNMConnectivity::ReconnectFeatures() and delete it via GNMConnectivity::DisconnectFeatures().

* If the graph is already built and we add some not connected features to the network we can anytime call AutoConnect() again and it will add additional connections not rebuilding old.

* The graph can be cleared anytime via GNMConnectivity::DisconnectAll().

* Virtual AutoConnect() method intends to be a generic for future derived classes for the concrete network formats. For example in GNM pgRouting Connectivity class it can call pgr_createTopology.

Convenient methods

GNMConnectivity::ConnectAllLinePoint() uses AutoConnect() but firstly counts all line and point layers, passing their ids as the parameters when the connection strategy is being created.

Week #7. Reviewed timeline

During this week I've:
- worked with python tests which I had to implement as it usual for testing GDAL. Unfortunately I didn't have enough time to fully implement them;
- chosen a final way of applying my changes in GDAL library: a fork of the official GDAL GitHub repo: https://github.com/MikhanGusev/gdal;
- added several small methods to GNM API which were committed with all my previous GNM work in the initial commit of MikhanGusev/gdal.

I also made a reviewed timeline which more strictly describes the rest of my summer of code, though it doesn't differ so much from the general plan. The main differ is that I'm no longer implementing the driver, while I implement the separate set of classes. This differ causes the shifting of the timeline. Also, some steps which I planed for the ending (like API-testing applications) I've already partly implemented, so the order of general plan steps now differs a bit.

1. (July 5 – July 11) Complete python tests of current functionality

1.1. API tests (GNMConnectivity, GNMManager)

1.2. Application tests (gnminfo)

2. (July 5 – July 25) Complete moving all functionality to new API in this order:

2.1. ~(July 5 – July 11) Automatic connectivity building

2.2. ~(July 12 – July 18) Features addition and deletion

2.3. ~(July 12 – July 18) Graph analysis

2.4. ~(July 19 – July 25) Network's business logic

3. (Also July 19 – July 25) Another console application which works with the new API.

4. (July 26 – August 1) Python tests of the rest of functionality and applications.

5. (Also July 26 – August 1) Documentation and tutorial.

6. (August 2 – August 11) Either create method smth like GNMConnectivity :: ImportFromPGRouting() or implement a full support of pgRouting as I described here.