sábado, 7 de julio de 2012

Using Django 1.4 with Eclipse (Part 2 Creating the Models)

I switched to WordPress. My new blog is at luiscberocal.wordpress.com. Part 3 of is at http://wp.me/p1JIuD-3F

This is part 2 of our tutorial on how to use Django with Eclipse. On Using Django 1.4 with Eclipse (Part 1 Configuration) we discussed how to configure Eclipse to work with Django and VirtualEnv.

On Part 2 we'll see how to create a model and synchronize it to the database.


The source code for this tutorial is hosted at https://github.com/luiscberrocal/django_eclipse_tutorial
I'm creating a tag for each part,  the tag for part 2 is https://github.com/luiscberrocal/django_eclipse_tutorial/zipball/Part_02_Creating_the_Models

Git Resources and Tutorials

I recently had to do some research about how Git worked. I saw on YouTube Introduction to Git with Scott Chacon of GitHub which was great for general concepts. But the one that answered most of my questions was this one by Patrick Hogan:

First of all finally figured how to do a fetch and merge instead of a pull.
git fetch

git merge origin/master
Now among the resources listed by Patrick:
  • Git Tower a mac graphic client for Git
  • Kaleidoscope an app to spot differences between files.
  • DiffMerge: an application to visually compare and merge files within Windows, Mac OS X and Linux.
  • Git Immersion: A very nice tutorial for git.
  • Gitolite: allows you to setup git hosting on a central server, with very fine-grained access control and many (many!) more powerful features.
Another great resource I found was Git – setting up a remote repository and doing an initial “push” where the author explains how to display the current branch on your terminal.

viernes, 29 de junio de 2012

Using Django 1.4 with Eclipse (Part 1 Configuration)

I used Python for my master's thesis project and honestly I didn't liked it. Recently I had a project at work and went with Python to solve the problems. In that process I discovered or actually rediscovered Django. So I decided to create a couple of tutorials to help mi get a grip of Django.

martes, 3 de abril de 2012

Suitability Analysis with ArcGIS (Part 3 Suitable Parcels)

On part 3 we are going to select the most suitable parcels for a waste disposal site.
Some of the tools we'll be using are:
1. Times
2. Raster to Polygon
3. Set Null
4. Select

miércoles, 28 de marzo de 2012

Suitability Analysis with ArcGIS (Part 2 Suitability Model)

On part2 of our tutorial we'll discuss how to create the suitability raster.
Among the Spatial Analyst tools that we'll be using are:
1. Reclass
2. Weighted Overlay

lunes, 26 de marzo de 2012

Suitability Analysis with ArcGIS (Part 1 Restriction Mode)

This is a tutorial on how to create a restriction model. A restriction model is a boolean raster of zeros and ones. Where a cell with zero value represents a cell restricted for the proposed use and a cell with value one represents a viable cell for the proposed use.
The tutorial is set to find a suitable area for a waste disposal site. Among the ArcGIS tools used are:
1. Union
2. Feature to Raster
3. Buffer
4. Is Null
5. Con

viernes, 23 de marzo de 2012

Using Zonal Statistics to Explore your Data with ArcGIS 10

I recently was asked by a student of mine for practical example on how to use Spatial Analyst's Zonal Statistic and I couldn't come up with anything practical. After a little research I decided to create this tutorial to show my students how use Zonal Statistics.

martes, 13 de marzo de 2012

Least Cost Path Tutorial using ArcGIS 10

Somewhere along the evolution of ArcGIS the Least Cost Path tool was eliminated from the Spatial Analyst Toolbar (as many other tools). I had not noticed this until recently, when I was preparing a class.

So I decided to create this tutorial in YouTube to help me and other remember how to create a simple least cost path using ArcToolbox.

jueves, 19 de enero de 2012

My First RESTful Web Service with Symfony2 (Pagination) Part II

On the first part My First RESTful Web Service with Symfony2 Part I. I discussed how to create a simple restful web service using JSON. I mainly focused on how to create the Symfony2 project and make it respond to json call.

On this post I´ll discuss how to make the web service paginate and create a simple interface to display the results using jQuery.

1   Changing the MRoute Repository

On Part I we laid the ground for pagination on the repository but didn´t work out the details.

What we're going to do is make pagination work and add the getCount method that will return the count of MRoutes on the database.

  1. Open srcMyGISGISBundleEntityMRouteRepository.php.
  2. Replace the previous getMRoutes method with the following. The previous code had a bug on the order by lines.
    public function getMRoutes($order_by = array(), $offset = 0, $limit = 0) {
        //Create query builder for languages table
        $qb = $this->createQueryBuilder('m');
        //Show all if offset and limit not set, also show all when limit is 0
        if ((isset($offset)) && (isset($limit))) {
            if ($limit > 0) {
                $qb->setFirstResult($offset);
                $qb->setMaxResults($limit);
            }
            //else we want to display all items on one page
        }
        //Adding defined sorting parameters from variable into query
        if (isset($order_by)) {
            foreach ($order_by as $key => $value) {
                $qb->add('orderBy', 'm.' . $key . ' ' . $value);
            }
        }
        //Get our query
        $q = $qb->getQuery();
        //Return result
        return $q->getResult();
    }
    
  3. On the same class add the following method:
    public function getCount() {
        $query = $this->createQueryBuilder('m')
                ->select('COUNT(m.id)')
                ->getQuery();
        return  $query->getSingleScalarResult();
    }
    

2   Changing the MRoute Controller

There are four things we need to do to the contoller. Read the count of MRoutes from the repository, Order the MRoutes by id, calculate the offset and define the routes.

  1. Open srcMyGISGISBundleControllerMRouteController.php.
  2. Replace the previuos routingInfoAction method with the following code:
    /**
     * Gets paged MRoutes JSON objects.
     * @Route("/routing-info/info.json",
     *     defaults ={"page" = 1, "count" = 10},
     *     name="mroute_json_list2")
     * @Route("/routing-info/{page}/{count}/info.json",
     *     requirements = {"page" = "\d+", "count" = "\d+"},
     *     defaults ={"page" = 1, "count" = 10},
     *     name="mroute_json_list")
     * @Method("get")
     */
    public function routingInfoAction($page, $count) {
        $offset = ($page - 1) * $count;
    
        $em = $this->getDoctrine()->getEntityManager();
         $total = $em->getRepository('MyGISGISBundle:MRoute')->getCount();
    
        $mroutes = $em->getRepository('MyGISGISBundle:MRoute')->getMRoutes(array('id' => 'ASC'), $offset, $count);
        $r_array = $this->routes2Array($mroutes);
        $r = array('page' => $page, 'count' => $count, 'total' => $total,'routes' => $r_array);
        return new Response(json_encode($r));
    }
    

    Now the web service will respond to http://localhost/mroute-rest-service/web/app_dev.php/mroute/routing-info/info.json using route mroute_json_list2 and will respond to http://localhost/mroute-rest-service/web/app_dev.php/mroute/routing-info/2/10/info.json will respond to route mroute_json_list.

Now we´re ready to consume the service.

3   Consuming the Web Service

Since I'm a little green using Twig what I did was create a new Netbeans PHP project and add on the index.html the following code:

<html>
   <head>
       <title></title>
       <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
       <script type="text/javascript" src="../js/jquery-1.6.4.min.js"></script>
        <link rel="stylesheet" type="text/css" href="../css/routing2.css" media="screen">
       <style>
           .page-link{
               margin: 0.2em;
               padding: 0.2em 0.5em;
               background-color: #1c94c4;
           }
           .page-link.selected{
               color: white;
               background-color: #005B81;

           }
           .page-link:hover{
               cursor: pointer;
           }
           .coordinates{
               height: 30px;
           }
       </style>
       <script type="text/javascript" >
           function roundNumber(rnum, rlength) { // Arguments: number to round, number of decimal places
               var newnumber = Math.round(rnum * Math.pow(10, rlength)) / Math.pow(10, rlength);
               if(rnum % 1)
                   return newnumber;
           }
           function showData(page, count){
               $('#routes').empty();
               $('#routes').html('<span class="loading">Loading ' + count + ' records for page ' + page +'...</span>');
               var baseUrl ='http://localhost/mroute-rest-service/web/app_dev.php/mroute/routing-info/'
                   + page + '/' + count +'/info.json' ;
               $.ajax({
                   url      : baseUrl,
                   dataType : 'json',
                   success  : function(data){
                        $('#routes').empty();
                       $.each(data.routes, function(i,route){
                           var li = $("<div/>").attr("id","route-" + route.id);
                           li.html(route.name);
                           $.each(route.points, function(i,point){
                               if(point.ptype == 'STOP'){
                                   var num = i +1;
                                   var className = 'stop-0' + num
                               }
                               var pointDiv   = $("<div/>").attr("id","point-" + point.id);
                               pointDiv.addClass('coordinates');
                               pointDiv.html('<span class="' + className + '" style= "float:left;"></span>' +point.x + ', ' + point.y);
                               li.append(pointDiv);
                           });
                           $('#routes').append(li);
                       });
                       var pagingDiv = $("<div/>").attr("id","paging-info");
                       pagingDiv.html('Page ' + data.page + ' count: ' + data.count+ ' total: ' + data.total + '<br/>');
                       var m = Math.ceil(data.total / data.count);
                       for(i =1; i <= m; i++){
                           var pageLink = $("<span/>").attr({"id" : "page-link-"+i, 'class' : 'page-link'}).html(i);
                           if(i == page){
                               pageLink.addClass('selected');
                           }
                           pageLink.bind('click',{'page': i}, onClickLink);

                           pageLink.appendTo(pagingDiv);
                       }
                       $('#routes').append(pagingDiv);
                   }});


           }
           function onClickLink(event){
               //alert(event.data.page);
               showData(event.data.page, 10);
           }

           $(document).ready(function(){
               showData(1,10);
           });

       </script>
   </head>
   <body>
       <div>
           <div id="routes"></div>
       </div>
   </body>
</html>

Now you´re able to consume the web service.

viernes, 13 de enero de 2012

My First RESTful Web Service with Symfony2 Part I

I want to create a simple Resftul Web service for reading data in JSON format. On part I I´ll discuss the basics on how to configure the web service on part II I´ll discuss how to introduce pagination to the web service.

Requirements:

  • PHP 5.3.6
  • Netbeans 7.0.1
  • Wamp 2.0

1   Installing Symfony2

  1. Download Symfony2 in zip format from http://symfony.com/download.
  2. Unzip the file. To your NetbeansProjects directory,
  3. Rename the symfony directory to mroute_rest_service.
  4. Open a command Window and move to the mroute_rest_service.directory.

    Run:

    php .\app\console --version

    Symfony version 2.0.9 - app/dev/debug

2   Creating the Bundle

  1. On the command window run:

    php .\app\console generate:bundle


Welcome to the Symfony2 bundle generator

Your application code must be written in bundles. This command helps
you generate them easily.

Each bundle is hosted under a namespace (like Acme/Bundle/BlogBundle).
The namespace should begin with a "vendor" name like your company name, your
project name, or your client name, followed by one or more optional category
sub-namespaces, and it should end with the bundle name itself
(which must have Bundle as a suffix).

Use / instead of \ for the namespace delimiter to avoid any problem.
  1. Fill as follows:

Bundle namespace: MyGIS/GISBundle

In your code, a bundle is often referenced by its name. It can be the
concatenation of all namespace parts but it's really up to you to come
up with a unique name (a good practice is to start with the vendor name).
Based on the namespace, we suggest MyGISGISBundle.

Bundle name [MyGISGISBundle]:

The bundle can be generated anywhere. The suggested default directory uses
the standard conventions.

Target directory [C:\NetbeansProjects\gis_rest_services/src]:

Determine the format to use for the generated configuration.

Configuration format (yml, xml, php, or annotation) [annotation]:

To help you getting started faster, the command can generate some
code snippets for you.

Do you want to generate the whole directory structure [no]?

Summary before generation

You are going to generate a "MyGIS\GISBundle\MyGISGISBundle" bundle
in "C:\NetbeansProjects\gis_rest_services/src/" using the "annotation" format.

Do you confirm generation [yes]?

Bundle generation


Generating the bundle code: OK
Checking that the bundle is autoloaded: OK
Confirm automatic update of your Kernel [yes]?
Enabling the bundle inside the Kernel: OK
Confirm automatic update of the Routing [yes]?
Importing the bundle routing resource: OK


You can now start using the generated code!

3   Generating Doctrine Entities

We are going to to have two entities Route and Points.

  1. On the command window run:

    php .\app\console gen:doctrine:entity

  2. On Entity shortcut name type MyGISGISBundle:MRoute
  3. Accept defualt configuration format annotation
  4. Add the fields as follows:
    1. Name

      Field name: name

      Field type: string

      Field length: 125

  5. On Do you want to generate an empty repository class answer yes
  6. On the command window run:

    php .\app\console gen:doctrine:entity

  7. On Entity shortcut name type MyGISGISBundle:Point
  8. Accept defualt configuration format annotation
  9. Add the fields as follows:
    1. Name

      Field name: name

      Field type: string

      Field length: 50

    2. X

      Field name: x

      Field type: decimal

    3. Y

      Field name: y

      Field type: decimal

    4. PType

      Field name: pType

      Field type: string

      Field length: 25

  10. On Do you want to generate an empty repository class answer yes

4   Opening the Project with Netbeans

Open Netbeans.

  1. Select File\New Project
  2. Select PHP Application with Existing Sources.
  3. Click Next.
  4. On the New PHP Application with Existing Sources
    1. Select the source folder where you unzipped the Symfony framework (C:\NetbeansProjects\mroute_rest_service).
    2. Select PHP Version: 5.3
    3. Click Next
  5. Click Finish.

5   Connecting to the Database

5.1   Connection Parameters

On Netbeans open C:\NetbeansProjectsmroute_rest_service\app\config\parameters.ini.

Edit the connection parameters as shown:


[parameters]
database_driver = pdo_mysql
database_host = localhost
database_port =
database_name = gisroutes
database_user = root
database_password =
5.2   Making the One to Many Relationship work on the Entities

I had to figure out how to se the relationship using annotations. I found an example in XML and adapted it [1]. Another thing I found out was that I had to manually set the precision and scale of the decimal fields.

  1. Edit the Point.php and add the route

    /**
    * @ORM\ManyToOne(targetEntity="MRoute", inversedBy="points")
    */
    private $route;

    ...
  2. On the Point.php edit the $x adn $y Column definitio to add precision and scale as show:

    /**
    * @var decimal $x
    *
    * @ORM\Column(name="x", type="decimal", precision=10, scale=3)
    */
    private $x;

    /**
    * @var decimal $y
    *
    * @ORM\Column(name="y", type="decimal", precision=10, scale=3)
    */
    private $y;
  3. Edit the MRoute.php and add the points

    use Doctrine\Common\Collections\ArrayCollection;
    ...
    class Route {
    ...

    /**
    * @ORM\OneToMany(targetEntity="Point", mappedBy="route")
    * @var ArrayCollection $points
    */
    private $points;

    ...

    public function __construct() {git

    $this->points = new ArrayCollection;
    }

    ...
  4. I found this cool hay to generate getters and setters [2]. The following command will create the getters and setters for the Point and Route entities.

    php appconsole doctrine:generate:entities MyGISGISBundleEntity

5.3   Creating the Database and Tables

To create the database run:

php app/console doctrine:database:create

To create the tables run:

php app/console doctrine:schema:update --force
5.4   Generating CRUD Controllers and Templates (Optional)

As far as the web service is concerned you don´t need t create the CRUD controllers. I´m creating them to add data.

  1. On the command window run:

    php .\app\console doctrine:generate:crud

  2. Fill like this:

    The Entity shortcut name: MyGISGISBundle:MRoute

    Do you want to generate the "write" actions [no]? yes

    Configuration format (yml, xml, php, or annotation) [annotation]:

    Routes prefix [/mroute]:

    Do you confirm generation [yes]?

    Confirm automatic update of the Routing [yes]?

Now we are ready to make our first test.

5.5   Viewing the Data on Apache

On Wamp´s Apache create an alias names route-rest-service pointing to C:\NetbeansProjects\mroute_rest_service.

Open a brower http://localhost/mroute-rest-service/web/app_dev.php/mroute/

6   Creating a RestFul Web Service

I found an example on how to configure the routing usting yml [3] and I adapted it to annotations.

What we want is to get a list of routes with their points in json format.

First thing we need to do is add a method that will fetch the entities from the database. The following code includes the pagination concept wich we will discuss on Part II.

  1. Open src\MyGIS\GISBundle\Entity\MRouteRepository.php
  2. Add the following code whitin the class

    public function getMRoutes($order_by = array(), $offset = 0, $limit = 0) {
    //Create query builder for languages table
    $qb = $this->createQueryBuilder('m');
    //Show all if offset and limit not set, also show all when limit is 0
    if ((isset($offset)) && (isset($limit))) {
    if ($limit > 0) {
    $qb->setFirstResult($offset);
    $qb->setMaxResults($limit);
    }
    //else we want to display all items on one page
    }
    //Adding defined sorting parameters from variable into query
    if (isset($order_by)) {
    foreach ($order_by as $key => $value) {
    $qb->add('orderBy', 'l.' . $key . ' ' . $value);
    }
    }
    //Get our query
    $q = $qb->getQuery();
    //Return result
    return $q->getResult();
    }
  3. Open src\MyGIS\GISBundle\Controller\MRouteController.php
  4. Add the following code:

    /**
    * Deletes a MRoute entity.
    *
    * @Route("/routing-info/info.json", name="mroute_json_list")
    * @Method("get")
    */
    public function routingInfoAction() {
    $page = 1;

    $count = 25;
    $em = $this->getDoctrine()->getEntityManager();
    $mroutes = $em->getRepository('MyGISGISBundle:MRoute')->getMRoutes();
    $r_array = $this->routes2Array($mroutes);
    $r = array('page' => $page, 'count' => $count, 'routes' => $r_array);
    return new Response(json_encode($r));
    }

    private function routes2Array($routes){
    $routes_array = array();

    foreach ($routes as $route) {
    $points_array = array();
    foreach ($route->getPoints() as $point){
    $p_array = array('id' => $point->getId(), 'name' => $point->getName(),
    'x' => $point->getX(),'y' => $point->getY(), 'ptype' => $point->getPType());
    $points_array[] = $p_array;
    }
    $r_array = array('id' => $route->getId(), 'name' => $route->getName(), 'points' => $points_array);
    $routes_array[] = $r_array;
    }
    return $routes_array;
    }
  5. You´ll have to add some data to the se the web service working. Since I haven´t figured out how to install the FixtureBundle from behind a proxy I add the data via sql using phpMyAdmin. Use the SQL in https://gist.github.com/1594781 to add data.
  6. Run http://localhost/mroute-rest-service/web/app_dev.php/mroute/routing-info/info.json from your browser and voila it works!!

7   References

[1]
  1. Association Mapping http://www.doctrine-project.org/docs/orm/2.0/en/reference/association-mapping.html#one-to-many-bidirectional
[2]Databases and Doctrine ("The Model") http://symfony.com/doc/current/book/doctrine.html
[3]Routing http://symfpony-project.org/

domingo, 1 de enero de 2012

Using restructuredtext in Blogger

Note

Updated 2012-0109: Added capability to clean_white_spaces.py to accept parameters

When I first blogged about restructuredText [1] I thought I had found a cool way to use TextMate for blogging.

My workflow was:

  1. Type the post in TextMate in restructuredText format.
  2. Run rst2thml.
  3. Open the html file
  4. Copy the tags inside <body> tag to Blogger.
  5. Edit the post in blogger to eliminate the white spaces.

This is not so bad. Until I wanted to update a rather long post I had. I realized that I would have to redo all my steps.

I had read about rst2wp which is a tool to convert to WordPress [2] and found out someone had adapted a script for blogger [3].

Installing rst2bs

These are the step I took to install rst2bs in my Mac with Lion Os X.

  1. Download the script from http://gitorious.org/mles/mles/trees/master by click on Download master as tar.gz
  2. Double click on mles-mles-master.tar.gz.
  3. Open terminal and run the following commands:

    sudo cp ~/Downloads/mles-mles/rst2bs.py /usr/local/bin/

    chmod 755 /usr/local/bin/rst2bs.py

  4. Test it

    rst2bs.py installing_pygments.rst installing_pygments_bs.html

Eliminating White Spaces

For that I wrote a simple Python script. Please bare in mind that my Python skill are almost non existent.

My clean_white_spaces.py reads:


import sys;

fd = open(sys.argv[1])
contents = fd.readlines()
fd.close()
new_contents = []

of = open(sys.argv[2], "w")
print "Eliminanting white spaces from " + sys.argv[1] + " to " + sys.argv[2]
# Get rid of empty lines
pre = False
line_num = 0
for line in contents:
line_num += 1
find_open_pre = "<pre>" in line
find_close_pre = "</pre>" in line
if find_open_pre:
print "PRE FOUND OPEN " + str(line_num)
pre = True
if find_close_pre:
print "PRE FOUND CLOSE " + str(line_num)
pre = False
if not pre:
# Strip whitespace, should leave nothing if empty line was just "\n"
if not line.strip():
continue
# We got something, save it
else:
new_contents.append(line.rstrip('\n'))
else:
new_contents.append(line)
of.writelines(new_contents)
of.close

# Print file sans empty lines
#print "".join(new_contents)
print "Finished"

This script will strip the white spaces if they are not within a <pre> tag.

Run it with the html generated by rst4bs.

python clean_white_spaces.py installing_pygments_bs.html installing_pygments_bs_clean.html

References

[1]Installing restructuredText Bundle for TextMate http://tecnofuenteabierta.blogspot.com/2011/12/installing-restructuredtext-bundle-for.html
[2]Using ReStructured Text with WordPress http://blog.mafr.de/2008/03/22/using-rst-with-wordpress/
[3]Some Twist on the Blog http://grissiom.blogspot.com/2009/12/some-twist-on-blog.html