Use Python to Keep Your Brain Sharp

“Use it or lose it” certainly applies where brain function is concerned. The experts tell you to exercise your brain to keep it in shape and ward off forgetfulness and possibly dementia when you are older.

One way to exercise your brain is to do computations in your head. Not long ago I read a book called The Power of Forgetting by Mike Byster that introduced several tricks and methods for doing these mental computations. One that stuck with me was how to multiply two, two-digit numbers in your head. Here’s an excerpt from the book explaining how to do it:

addnumbersThere are probably easier ways to do mental multiplication but Byster’s method is meant to be an exercise in remembering and forgetting numbers at will to make your brain stronger.

I wanted to find a way to challenge myself with the above multiplication method on a regular basis. My solution was to write a very small Python script that would generate random two-digit by two-digit multiplication problems and display them on screen. Here is what I came up with:

from random import randint
from ctypes import windll

firstNumber = randint(10,99)
secondNumber = randint(10,99)

problem = str(firstNumber) + " x " + str(secondNumber)
answer = str(firstNumber*secondNumber)

windll.user32.MessageBoxA(0, problem, "Can you multiply these in your head?", 0)
windll.user32.MessageBoxA(0, "The answer is: " + answer, "How did you do?", 0)

The script simply generates two random two-digit numbers, displays them to the user, then displays the correct answer when the user closes the first message box. I used Python’s ctypes library to create Windows message boxes,so you would need to make some adjustments if you wanted to use it on a different operating system. I also set up my task scheduler to run the script automatically every hour. Now, throughout the day at work and at home I’m reminded to exercise my mind in a way I wouldn’t normally exercise it.

Efficient ArcServer Cache Management with a Staging Server

Getting a cache built can sometimes be a challenge but caching an ArcServer map service can be important if you want your web apps to display fast. Depending on the subject of your cache and what scales you cache at you can end up with tens or hundreds of gigabytes of data.

Art Tiles

In my office we cache both vector and imagery data for an entire county. We have 20 aerial mosaic data sets dating back to 1937 and several vector data sets that also cover the entire county. Caching all of this can put a real strain on a server and takes up limited server resources that can cause the server to perform poorly.

I’ve found that the best solution is to cache on a staging server and then transfer the cached tiles to a prepared production server. Here is how you can easily do the same:

  1. Create a service on your staging server.
  2. Set up caching in the Services Properties dialog.
  3. Create a service on your production server that has the same name as your staged service.
  4. Set up caching in the Services Properties dialog. From the “Tiling Scheme” drop down select “An existing cached map / image service” and navigate to the staging service. This will import all of your cache advanced setting that you defined on your staging service like scale levels.
  5. Run the Manage Map Server Cache Tiles tool to create the staged cache.
  6. Now you just want to copy your level folder (L00, L01, L02…) located in \arcgisserver\directories\arcgiscache\[your map service name]\_alllayers to the same location on your production server.

At this point your production server should have a full working cache. If you have a cache service that will need to be updated regularly you could even script the whole process and schedule it to run at night or on the weekend.

 

How to Think Like Sherlock Holmes

Mastermind

I often go to the library to try and find something entertaining, intriguing or just different. Last week I found just the thing. It was a book called Master-Mind: How to think like Sherlock Holmes by Maria Konnikova. Now this is the kind of book I love to find. After all, who wouldn’t want to be able to think like the famous detective of 221B  Baker Street?

The title itself intrigued me enough to take it home right away. It promised to free my mind from being common and show me how to be a logical processing machine. Perhaps I could become as great a thinker as Holmes was portrayed to be in his stories.

I won’t go into a deep book review here. If you want to know the details you will have to read it yourself.  I will tell you, however, that after reading the entire thing, my brain is not yet governed by a Holmes-like system of thinking. I have not yet mastered the skills of observation, imagination or deduction that it takes to become a master thinker. But I am further along that road than before reading Mastermind. Many of the book’s insights have actually already helped me solve problems with my programming using the methods that Holmes used to solve his own puzzles.

The book was a fun read. I would encourage you to buy a copy or go to the library and check it out.

Better Basemap Management for the ESRI Javascript API

If you use ESRI’s Javascript API to build web maps you have probably noticed that your ability to manage base map layers is somewhat limited. You really only have three out-of-the-box alternatives:

  1. Define an ArcGIS.com basemap in the map constructor
  2. Create and fill a Basemap Gallery
  3. Switch between two basemaps with Basemap Toggle

But what if you don’t want to use an ArcGIS.com basemap, you want to use something other than thumbnail images to switch between basemaps or you have more than two basemaps to switch between? In that case, you probably need something a little more flexible.

I recently found myself in one of those situations. I wanted to be able to switch between a vector basemap and a basemap of current imagery as well as give viewers the opportunity to choose imagery as far back as the 1930s. I really didn’t want to show thumbnails for every year of imagery. Instead, it seemed to make more sense to allow selection by year from a drop down menu.

So I went ahead and built my own little version of a basemap switcher to handle different situations. To be fair, I could have used the ESRI Basemap Toggle dijit to handle the two main basemaps but I like having more control over my code than Dojo based dijits allow.

I put together a simple map to demonstrate switching multiple basemaps through various means like buttons, thumbnails and drop downs. The html markup and CSS  styling is pretty basic. We create a header as a toolbar (in this case our switcher is the only tool), a div for the map to be injected and markup for the the jQuery UI dialog box that will hold our switching controls.

Edit: FC Basson’s comment got me thinking about the code samples here being too complicated for what they were actually doing. I went back through and removed the toolbar and all jQuery UI and jQuery code. None of these added anything special to the app and since Dojo is already available to us, it should be used instead

<!doctype html>
<html lang="en">
<head>

    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=7, IE=9, IE=10">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <meta name="apple-mobile-web-app-capable" content="yes">
    <meta name="apple-mobile-web-app-status-bar-style" content="translucent-black">

    <title>Custom Basemap Switcher</title>

    <script src="http://js.arcgis.com/3.7/"></script>

    <link rel="stylesheet" href="http://js.arcgis.com/3.7/js/dojo/dijit/themes/nihilo/nihilo.css">
    <link rel="stylesheet" href="http://js.arcgis.com/3.7/js/esri/css/esri.css">

    <style>
        html, body, #map
        {
            height: 100%;
            width: 100%;
            margin: 0;
            padding: 0;
            overflow: hidden;
        }

        #map
        {
            position: fixed;
        }

        #basemapButton
        {
            position: absolute;
            top: 25px;
            left: 70px;
            z-index: 100;
        }

        button
        {
            border-radius: 4px;
            background-color: #F1F1F1;
            font-size: 1em;
            color: #33A7DE;
        }

        #basemapDialog
        {
            background: white;
            display: none;
            font-size: 0.75em;
            height: 200px;
            left: 65px;
            position: absolute;
            top: 70px;
            width: 300px;
            border: 0.5px solid black;
            border-radius: 5px;
        }

            #basemapDialog ul, li
            {
                list-style: none;
                margin-top: 1em;
            }

        a img
        {
            border: 1px solid black;
        }

        a:hover img
        {
            border: 1px solid white;
        }
    </style>


</head>
<body class="container tundra">
 <button id="basemapButton" title="Open a basemap switching dialog">Switch Basemaps</button>
 <div id="map"></div>
 <div id="basemapDialog" title="Choose a Basemap">
 <ul>
 <li>
 <button id="wldImagery" title="Load the World Imagery layer as a basemap using this button">Imagery</button> Select by button</li>
 <li><a id="streetView">
 <img title="You can use an image thumbnail like this one to select a basemap" src="http://ryanrandom.com/examples/images/vegas.jpg" alt="Street Map View" height="50" width="50" />
 </a> Select by thumbnail</li>
 <li id="differentBase" title="View historical imagery as a basemap">
 <select name="bmSelect" id="bmSelect" title="Selecting a basemap from a dropdown is handy when you have many choices, like historical basemaps">
 <option value="none" selected>None Selected</option>
 <option value="topo">Topo</option>
 <option value="relief">Shaded Relief</option>
 </select> Select by dropdown
 </li>
 </ul>
 </div>
</body>
</html>

The code needed to run the switcher isn’t too complex either. You just need to create an ESRI Javascript map reference, create variable references to the basemaps you want to use then create a function that does the actual changing of the layers.  The rest of the code is just some jQuery Dojo/JavaScript to interact with the controls.

    <script>
        require([
            "esri/map",
            "esri/layers/ArcGISTiledMapServiceLayer",
            "esri/geometry/Extent",
            "dojo/dom",
            "dojo/dom-attr",
            "dojo/dom-prop",
            "dojo/on",
            "dojo/query",
            "dojo/NodeList-traverse"
        ], function (
            Map, ArcGISTiledMapServiceLayer, Extent, dom, domAttr, domProp, on, query
          ) {
            //We can set an extent like this one which takes us to Las Vegas, NV
            var initialExtent = new Extent(-115.68, 35.77, -114.75, 36.45);

            //Create an empty array to store a reference to the currently selected basemap layer
            currentBasemap = []

            map = new Map("map", {
                extent: initialExtent,
                logo: false,
                slider: true
            });

            //create references to all of your basemap layers but only add one to the map initially
            imagery = new ArcGISTiledMapServiceLayer("http://services.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer");
            //add the current basemap layer to the currentBasemap array so you can keep track of what is active.
            currentBasemap.push(imagery);
            map.addLayer(imagery);
            //call reorderLayer on the map object and set your basemap to zero every time you change basemaps. This ensures that your basemap is always the bottom layer.
            map.reorderLayer(imagery, 0);

            street = new ArcGISTiledMapServiceLayer("http://services.arcgisonline.com/ArcGIS/rest/services/World_Street_Map/MapServer");
            topo = new ArcGISTiledMapServiceLayer("http://services.arcgisonline.com/ArcGIS/rest/services/World_Topo_Map/MapServer");
            relief = new ArcGISTiledMapServiceLayer("http://services.arcgisonline.com/ArcGIS/rest/services/World_Shaded_Relief/MapServer");

            //Opens and closes the basemap dialog using the basemaps button
            on(dom.byId("basemapButton"), "click", function () {
                if (!(dom.byId("basemapDialog").style.display == 'block')) {
                    dom.byId("basemapDialog").style.display = "block";
                } else {
                    dom.byId("basemapDialog").style.display = 'none';
                }
            });

            //This function does just what its name says and changes the layer
            function changeLayer(layerName) {
                map.removeLayer(currentBasemap[0])
                currentBasemap.length = 0;
                currentBasemap.push(layerName)
                map.addLayer(layerName)
                map.reorderLayer(layerName, 0);
            }

            //Here is where we switch the basemap using a button
            on(dom.byId("streetView"), "click", function () {
                domProp.set(query("#differentBase option")[0], "selected", true);
                changeLayer(street)
            });

            //Here is where we switch the basemap using a thumbnail image
            on(dom.byId("wldImagery"), "click", function () {
                domProp.set(query("#differentBase option")[0], "selected", true);
                changeLayer(imagery)
            });

            //Here is the code for handling changing your basemap based on a html select box dropdown.
            dom.byId("bmSelect").onchange = function () {
                var newBasemap = dom.byId("bmSelect").value

                //Default to the imagery basemap if "none Selected" is the choice in the dropdown 
                if (newBasemap === "none") {
                    changeLayer(imagery)
                } else {
                    //The following allows you to map the dropdown values with an object reference to the tiled service layers created above
                    var bmList = ({
                        "topo": topo, "relief": relief
                    })

                    //Find the selected basemap in the array above and use the associated object to add the selected theme layer to the map
                    for (var x in bmList) {
                        if (x === newBasemap) {
                            changeLayer(bmList[x])
                        }
                    }
                }
            }
        });
    </script>

You will notice in the code above that we have create an empty array called currentBasemap.  This array is the key to making the tool work. We store a reference to the currently loaded basemap here. That way we can access it, remove it’s corresponding basemap and load a reference to whatever new basemap that we want to display.

That’s all there is to it. You can get the full code from GitHub. You can also view a working example of the code.

Let me know if you have any suggestions on how to make it better.

Master Google Maps, Research WWII and Convert Data

As I tooled around the web last week looking for anything useful, interesting or entertaining I came across the following four pages that I thought I would share. Hope you like them.

1. Gizmodo.com – 10 Tricks to Make Yourself a Google Maps Master

Google Maps MasterA great up-to-date look at Google Maps and what it takes to master its ins and outs. The bulk of the article deals with using Maps’ search box. That’s great because the search box is full of great functionality that most users probably don’t have experience with. No matter what your level of skill with Google Maps, there is probably a trick or two to learn here.

2. Worldology.com – World War II: Interactive Map

World War II Map

This is a great little Flash based map that gives you an interactive look at the progression of World War II in the European theater. You can visually see the progression of the war and you can read the popup excerpts as you mouse over the different countries. There is even an option to see current political boundaries overlaid on the map. Cool.

3. freegeographytools.com – Download Page

Free stuff is really neat. Free GIS stuff is even neater. Freegeographytools.com is a nifty little blog that focuses on, you guessed it, free geography tools. The download page provides links to many of these tools like data conversion utilities, KML tools and raster tools. It doesn’t seem like the blog has been updated since 2011. There might be better alternative out there but there is still some good information here.

Five Things I Love About the ESRI Javascript API

Having worked with both the ArcGIS Silverlight and Javascript APIs I have to say Javascript has been much more fun. There is just something about working on the front end of a web map and being able to have it do so many amazing things. Here are my top five reasons I enjoy working with this API.

1. Your maps are device independent. Unlike the Flex and Silverlight APIs, Javascript and HTML5 can be viewed on almost any device. The obvious advantage here is that you can reach more people with your spatial data.  Of course having the ability to code for multiple devices means having to customize your code for different screen sizes which can turn into a lot of work. Check out this post from the Treehouse Blog for a great primer on responsive web design.

2. Plenty of code samples and live examples. Code samples are a great way to get your own map up and running quickly without having to figure everything out first. While the samples are not perfect they are a good way to discover map functionality that you might want to incorporate.

3. Support for a ton of layer and data formats. From GeoRSS and GeoJSON to KML and shapefile, this API pretty much covers it.

4. Well written API reference documentation. If you are going to use an API (any API) you want to be able to understand it so you can use it effectively. Good documentation is critical for this. The Esri JS API is (in my own humble opinion) extremely easy to read and understand. Each class has all the information you need to use it without having to decode any meanings. Furthermore, there are links from each class to samples that use it.

5.  Editing tools for online editing of SDE data. Let’s be honest, online editing through a web browser is not a great idea if you are trying to build accurate data. However, there is definitely a place for it. If you are trying to outline an area inhabited by a herd of elk or mark an area of weed infestation, browser editing is probably fine.  It is great just to be able to have this ability so more diverse mapping applications can be developed.

Wow, this list was a lot harder to write than 5 Things I Hate About the ESRI Javascript API. One reason is that there are other APIs out there that do a lot of what the ESRI API does (yes I am thinking about OpenLayers). Another reason is that as I write I think about things I would like ESRI to do better or differently. Oh well, you can’t have everything. Let me know what your thoughts are on the ESRI Javascript API – good or bad.

5 Things I Hate About the ArcGIS Javascript API

Building a mapping application with ESRI’s Javascript API is a lot of fun. But it can also be a real pain. Here are five things that absolutely drive me crazy when writing an ArcGIS Javascript map.

1. It is based on Dojo. It’s probably just my lack of Javascript programming sophistication but I find Dojo a little hard to work with. Although the API documentation has improved in recent versions, I don’t find it very intuitive. If you are a beginner or new to Javascript, the getting started documents are just plain frustrating. Ben Farrell thinks there are seven stages of learning Dojo including anger and acceptance. I’m not sure if there are really only seven stages but I can attest to the anger and acceptance.

2. It uses multiple versions of Dojo. Here I go again with the Dojo thing. Half of the developer samples on the ArcGIS.com developer’s site seem to be written pre version 1.7 while the others are post 1.7 and rely on the AMD module format.  I understand that version transitions of supporting libraries can be difficult but this is ridiculous. Since the ArcGIS API updated to version 3.6, many of the developer samples have been updated to show the AMD require format. Unfortunately, the transition has been slow and there is still a lot of mixed version code and a lot of live samples are broken. If you are going to upgrade to a new version of your API, let’s have things ready to go.  

3. No built-in label engine for dynamic, on-screen annotationIt seems obvious to me that the users of a map would want to mark it up with text; especially in conjunction with a draw toolbar. Apparently it isn’t obvious to the ESRI developers. Instead we are left to work with with the labelPoints function which uses the geometry server to create an unseen point and then puts a text symbol on top of it.

4. The label engine problem above leads me to my next gripe. The LabelPoints function doesn’t allow line breaks. That means that if you want to label a polygon with something like, oh say, area and perimeter, you have to have it all on one line or call the labelPoints function twice and use an offset. How about we get the ability to at least use a newline character (\n) and be done with it.

5. No out-of-the-box support for a table of contents. There is a lot of debate about whether an ArcGIS-like table of contents is a good thing in a web map. No matter what side of the debate you fall on, the fact is that people are used to using them and users often want the ability to turn layers on and off. There are a few projects out there that attempt  to solve the problem but each has its own limitation and problems.  It is a little surprising that ESRI doesn’t have a solution for this one yet. Or, maybe it isn’t really that surprising.

I have plenty more gripes with the Javascript API than the ones listed above but I don’t want to get too depressing. Also, I actually enjoy developing with the API (most days) and there are some great things about it. If you want to add to my list put it in the comments.

CSS Shorthand for Clean Web Map Styles

I see a lot of CSS code written inefficiently and sloppily, even on official and professional sites. It’s not that the code is wrong, it just could be much better. There is really no reason to have to write four lines of code to set a div tag’s margins or style a tag’s background. You can simply use CSS shorthand to get it all into one compact line.

The reason many people don’t use shorthand is because they’ve never taken the time to learn it. Instead, they lean on verbose style rules that spell everything out.   Below you will find a few common examples of CSS shorthand code that might help you when writing your CSS code. This is in no way a definitive guide. It is simply some of the more common properties you will encounter. You can also download a brief CSS shorthand cheat-sheet.

Margin and Padding:

There are four ways to define the four margins and padding of a CSS box.

  1. margin: [top] [right] [bottom] [left];

    Using four values defines each side individually. For example, margin: 1px 2px 3px 4px; would give an element such as a <div> box a top margin of 1px, a right margin of 2px, a bottom margin of 3px and a left margin of 4px.

  2. padding: [top] [right/left] [bottom];

    This notation relies on the same concept as above but now we only need three values to define our padding. padding: 3% 2% 3%; would give our html element a top and bottom padding of 3% and a right and left padding of 2%.

  3. margin: [top/bottom] [right/left];

    Here we only need two values to define our margins. margin 0 auto; gives a zero margin to the top and bottom and centers the <div> within its containing element.

  4. padding: [top/bottom/left/right];

    A single value defines all sides of an element equally. padding: 1em; applies a 1em  pad to the entire html element.

 

Background:

The background property can be defined by this notation –

background: [color] [url(image url)] [repeat] [attachment] [position];

The color property can be defined using hexadecimal(hex) notation, rgb or color names. The image property is written with the prefix “url” followed by parentheses containing the url to the image you want to use for your background. Repeat is used to tile the background image. If you use attachment you can control whether the background image scrolls with the page or not. Position tells the background image what position to start at on the page. The following is an example of a background:

background: #ffffff url(‘greatpic.gif’) repeat-x fixed center;

 

Font:

The font property can be demonstrated by-

font: [size] [weight] [style] [family];

Font size can be defined with a number and a unit such as px, % or em or with size value words like small, medium, large, etc. Weight is simply the thickness of the font characters. Style refers to the values  italic, oblique, normal or inherit. The family refers to the font family like Serif, Georgia or Helvetica. Here is an example:

font: 1em normal italic Helvetica;

 

Border:

Border properties are relatively simple-

border: [width] [style] [color];

The border of an html element is easy to style. You just need to designate the width, color and style. Width can be written using pixels or the pre-determined width designations thin, medium and thick. Style refers to values like solid, dotted and dashed. Color is written the same as in the background property and can be a hex number, rgb numbers or a written color.

border: 2px solid blue;

 

There are a lot more shorthand properties for CSS. Even some of the ones I posted above have further properties that could be included. For a more complete look at CSS shorthand methods check out this page from Mozilla.

Jump Start Your Geospatial Career with This Free E-book

It’s a great time to be a GIS professional. There are no shortages of jobs and the field keeps changing and expanding to include amazing technologies and applications. However, with increasing demand often comes increasing competition. Anyone who wants to enter the geospatial field or advance in it needs to pay attention to fast changing industry needs and make sure they are positioned to fill those needs.

The folks over at DirectionsMag.com have put together a collection of articles dealing with topics of interest to all levels of the geospatial community. It is called GIS Jobs, GISP Certification and Geospatial Careers and can be downloaded at

 

**

Update:  The following page has been removed from directionsmag.com

**

http://www.directionsmag.com/books/gis-jobs-gisp-certification-and-geospatial-careers/347075.

It is a great little PDF e-book covering things from GIS job interviews to issues surrounding certification. Have a look, it’s free.

 

Easily Export ArcGIS Attribute Table to CSV

For many ArcGIS users, exporting an attribute table to a .csv file or excel format file is a common part of their workflow. Unfortunately, exporting to either of these formats has never been a core functionality of ArcGIS. To overcome this you typically have had two choices: write a script using third party libraries or go through multiple steps exporting to a .txt file and converting it to .csv or .xls.

However, there is a little shortcut you can take to quickly export your data to .csv right from the attribute table.

  • From the table, click Export.
  • Under “Output Table” browse to the folder you want to put your .csv file in, save type as .txt and enter a name for your output making sure to leave off the ‘.txt’ from the end of the name.
  • If the ‘.txt’ extension is present in the name the tool will output a normal text file of your data. If, however, your name does not include the ‘.txt’ extension then your output will be a .csv file by default.