Managing your Web GIS using the ArcGIS Python API

The GIS industry has really come into its own in the last few years and is continuing to experience phenomenal growth rates. A PR Newswire report in early 2016 had the global geographic information system market growing at a compound annual growing rate of 11% between 2015 and 2020, mainly due to increased urbanization and the development of smart infrastructure. Naturally, this is expected to raise demand for skilled GIS professionals to run these systems and consequently there will be many individuals gunning for these positions. One sure way for a GIS professional to stand out is possessing some programming skills. If you’ve been in the GIS world for any amount of time, you’ll notice that the predominantly popular programming language is Python – a dynamic and strongly typed programming language designed to emphasize usability.

Esri has been offering support for Python from version 9.0. Since then, it has been accepted as the scripting language of choice for ArcGIS users and continues to find use in automating repetitive tasks in GIS analysis or map production. However, as Esri (and indeed, the world) moved towards the GIS-as-a-Service paradigm, Python hasn’t really received the attention it deserves as a development language for the web portion of the ArcGIS Platform. Granted, Python is a very powerful language and has been able to circumvent this lack of attention using packages such as ArcREST. This has changed at last with the release of the ArcGIS Python API, which was released as part of ArcGIS 10.5 in late December.

So … what is the ArcGIS Python API?

It is the Python based programming interface to your Web GIS, which could be an instance of ArcGIS Online or an enterprise implementation of ArcGIS (an ArcGIS Server fronted with a Portal for ArcGIS). With the introduction of Web GIS, the ArcGIS platform is now available in a wide variety of environments and devices because it exposes much of its functionality through web services. To develop with these web services, you use the ArcGIS REST API which allows you to:

  • Consume ready-to-use ArcGIS Online services hosted by Esri
  • Consume services published by you or by other organizations
  • Publish your own web services
  • Create and share items on ArcGIS Online or your own portal
  • Configure and automate parts of the ArcGIS system, such as ArcGIS Server and Portal for ArcGIS

The ArcGIS Python API provides an interface for these underlying web services, enabling Python´s programming language capabilities to be used over the web through an interactive console that bears an uncanny resemblance to the current Python window in ArcGIS Desktop and Pro. Not only does the API cater to current Python users – who utilize its scripting capabilities for data crunching or map design – the API also allows for GIS visualization and analysis, spatial data/content management as well as organization administration. The ArcGIS Python API automates, controls, drives and is powered by your Web GIS. It can be installed on any computer and works with both ArcGIS Online and ArcGIS Server via REST.

Who would need to use this API?

  • Web GIS Administrators (ArcGIS Online or Portal for ArcGIS users with administrative rights) – for purposes of automating repetitive tasks such as creation of users and assigning of roles and applying access restriction policies
  • Content publishers in instances of ArcGIS Online or Portal for ArcGIS – automated content creation and validation
  • Analysts or data scientists – for purposes of taking advantage of the analytical capabilities of the ArcGIS platform using Python’s rich set of libraries
  • Python programming enthusiasts – provides a gentle introduction to GIS workflows

How can I get started?

The API is meant for anyone with access to a Web GIS (either through an ArcGIS Online organization or an ArcGIS Server fronted with a Portal for ArcGIS). This means that even though you could work as an anonymous user of ArcGIS Online, your best use of the API will come through the use of an organizational ArcGIS Online account or an ArcGIS Enterprise implementation. To grab hold of one of these instances, get a free developer account so that you can get started quickly with free credits, features like geocoding, routing and geo-enrichment and premium content like demographic and traffic maps from Esri.

Setting up your environment

The install and setup for the API can be found here. To use version ArcGIS Python API, you need to use Python 3. In the example that follows, we will be using Jupyter – a web application that allows you to create and share documents that contain live code, equations, visualizations and explanatory text. To use the map widget for Jupyter notebook, you need Python 3.4 or 3.5, and ipywidgets version 5.1.5 or newer.

Creating your first Notebook

If your installation went through fine, you should be able to start up a Jupyter notebook by running the following the command ‘jupyter notebook’ on your terminal (i.e. command prompt on Windows) as shown below:

The command should consequently launch the following on your default browser:

 

From this interface, you can create a new Python 3 notebook as shown below:

 

This action will open a new page as shown below:

 

We can now start using the API interactively through the notebook.

The ArcGIS Python API is designed to be a Pythonic representation of a Geographical Information System. Simply put, this means that the API follows the conventions of the Python community and uses the language in the way it is intended to be used if a GIS were to be created using Python constructs. Consequently, this means that the API is also modular – with each module having a functions and types that cater to a particular aspect of the GIS. The diagram on the right is a visual representation of this.

The gis module provides an entry point to the GIS and allows one to manage groups, users and content in the GIS. It is essentially an information model for a GIS hosted within ArcGIS Online or an instance of ArcGIS Enterprise hosted in your premises. Thus, to get started, we will invoke this module as shown below:

 

The code above, essentially imports the GIS object from the gis module in the first line, then creates an instance of the GIS object named devGIS in the second line. To instantiate a GIS object from an organizational account (such as your newly created developer account), you need to pass in the URL of your account, and login credentials as shown in the image above. You may, however, choose to instantiate the GIS object as an anonymous user and in that case may instantiate as shown below:

RELATED:  The Present and the Future in Geospatial technology: What transformation are we looking forward to seeing in this industry?

 

The third line instructs the notebook to visualize what has just been created.

In order to run the code, click on the ‘run’ symbol: or maneuver as shown below:

 

The following should be your result:

 

The output proves that we have successfully logged on to our GIS and that we are currently in a map viewer hosted ion the GIS. The viewer is interactive and one can zoom in or out of the viewer’s current extent. At this point, we can save our notebook, just so that if we close it by mistake we can continue from where we left off. Click on the File menu and proceed to save using the Save and Checkpoint menu item. You can then use the Rename option to rename the file to your name of choice. Once done, we can now go on with our demo.

To continue coding, if you find that you have no cell to write code in, add a new one using the Insert menu as shown below:

 

We may want to add some data to our current map. Let’s have a look at some crime data which is freely provided to us by the ACLED (Armed Conflict Location & Event Data Project) website. Visit the site and download any CSV data file. We will add the csv data to our visualization using the ArcGIS Python API.

(PS: At present, one cannot import a file with more than one thousand records so if your file contains more, kindly whittle it down to a thousand or upload in parts)

The following code is used to read data from the CSV:

 

The first line of code allows us to make use of the pandas Python library to easily handle data in tabular formats.

The second line of code, uses an instance of the pandas library to read a CSV document that I have stored in the location provided and asks the pandas library to note that the encoding of the file is different from UTF-8.

The third line of code instructs the console to display what has been read by the library.

Click on the ‘run’ symbol: to obtain your output. The following should be the case – your data displayed on the browser:

 

Now, that we have read the data from the csv file, we need to import it into the map and may also need to save it for viewing purposes later. To do this, we will need to add the csv file as a content item in our GIS. We will once again call on the gis module to do this.

 

The code above does the following:

  • First line – shows the path of the csv file within the computer
  • Second line – Provides basic metadata about the item which we wish to upload to the GIS instance
  • Third line – shows the path to the thumbnail picture that will be used to represent the uploaded data item
  • Fourth line – calls on the content property of the GIS object in order to access the add() method which accepts a dictionary containing the properties of the item to be created.
  • Fifth line – instructs the console to display the created item

This code should have the following output:

 

This shows that the content has been uploaded. We publish this content as a layer on our GIS. This is achieved by running the code below:

 

This code ensures that the CSV is translated into a web service and visualizes the output Feature layer item on the console under the variable name ‘crime_feature_layer_item’:

 

We can now add this feature layer to our GIS Map Viewer by running the following code:

 

The code above opens another mapviewer, only this time the viewer’s extent covers Africa at a zoom level of 3. The second line utilizes the add_layer() method of the map object to add the layer we created earlier to the map viewer. Finally, the third line displays the new map viewer. The result is as follows:

 

An interactive map with the layer added on to the map viewer is the outcome. One can also choose how the data will appear once it is uploaded by using the following code to add the layer rather than the one we used before:

 

In the code sample above, the add_layer() method is made use of whilst utilizing the renderer and field name options. The renderer determines what symbology type will be applied to the layer and the field_name option provides the field whose values will be used to determine the size or color of the symbol for each feature in the layer.

Thus, we can see how we can add data to a GIS through the Python API with a few lines of code and show the steps taken through this notebook. The steps we have followed here can be packaged as a script with parameters and ran using a Task Scheduler to fully automate the task.

In later posts, we will see how we can go farther with data that we have uploaded and conduct some analysis on it using the API. In so doing, we will be able to have a look at more of the modules that the API is built from.

Till then, please take some time to visit the ArcGIS Python API site and get started on using it to write scripts that do all sorts of awesome things!

 

[Full disclosure: Laban Karanja is currently employed by Esri Eastern Africa. That said, the opinions expressed here are his own. ]