ArcGIS vs QGIS etc Clipping Contest Rematch revisited

Earlier this Last year, in June, Don Meltz wrote an interesting blog “ArcGIS vs QGIS Clipping Contest Rematch” where he let compete ArcGIS and Quantum GIS in a clipping contest. The benchmark contest data set in question is a 878MB ZIP file (ContourClipTest.zip). The blog page gained a lot of comments, even from ESRI since some ArcGIS versions crashed on this test data set.

Find below the various timings compiled from the blog and the comments:

1) Proprietary software

Software Processing time Hardware/Software
ArcGIS 9.3 crash after 1h 9min: ERROR 999999: Error executing function. Invalid Topology [4gb file limit.] Failed to execute (Clip) unknown
ArcGIS 10.0 crash likewise unknown
ArcGIS 10.1 ESRI promise to calculate it in 34 seconds in this updated version (did anyone test?) unknown
GlobalMapper (version?) 30 mins unknown
GlobalMapper v11.02 49 sec Windows XP w/ 3.5GB RAM
Manifold 8 (64bit) 31 min Windows XP64 16 gb. RAM and 2.33 GHz

Note: The two GlobalMapper results are a bit funny, perhaps always minutes?

2) Free and Open Source Software

Software Processing time Hardware/Software
Quantum GIS (version?; Simple features) 4-5 min unknown
GRASS GIS 7 (topological GIS) 5 min Dell PowerEdge 2950 from 2008, Intel Xeon 2.66GHz, 8GB RAM
gvSIG to be done
PostGIS to be done

Notes: Hope volunteers will test this also on gvSIG and PostGIS (and other FOSSGIS)! Please report…

 

17 comments on “ArcGIS vs QGIS etc Clipping Contest Rematch revisited

  1. Just to add to this…
    Cadcorp SIS MapModeller 8.0
    3 minutes 40.
    Dell Core2 laptop that at the time was being used as a web server, had chrome open and was being used for email. Will post the result later when this test is carried out on a proper desktop.

  2. Anonymous on said:

    You mean earlier last year (2011).

  3. JGRASS benchmark here: GIS clipping benchmark: jgrasstools & uDig (on JTS & geotools):

    http://jgrasstechtips.blogspot.it/2012/11/gis-clipping-benchmark-jgrasstools-udig.html

  4. Anonymous on said:

    PC: Intel core i7, 4 GB RAM; Win 7, 64-bit

    gvSIG 1.11 – 3-4 min.
    gvSIG 1.11 with SEXTANTE – 2-3 min.

    QGIS 1.8 – 4-5 min.

    MapWindow GIS 4.8.6 – 12 min.

  5. My two cents.

    Intel Core2 Quad CPU Q6600 @ 2.40GHz 64 bits; 4 GB RAM; Kubuntu 12.04 Kernel 3.2.0-32-generic

    QGIS 1.8
    Loading shapefiles: 3:35 min.
    Clipping: 3:38 min.

    gvSIG 1.12
    Loading shapefiles: 0:20 min.
    Clipping: 3:27 min. (core geoprocess)

    OpenJUMP 1.5.2
    Loading shapefiles: “Out of Memory Error”

  6. Sorry for being carelss with using the word “benchmark”, of course it is a “contest” as stated in the title. It may become a benchmark when executed on the same machine with most software packages in question being installed on it.

  7. Just tested on my new laptop (i7, 8gb ram) with Windows 64 and qgis-dev installed with osgeo4w.

    It tool 0:30 to open the shape and 2:50 to do the clip.

    I will try test also using Linux, GRASS and other clipping tools available in the Sextante toolbox.

  8. and the winner is …

    JTS/GEOS because a number of different applications could successfully use JTS/GEOS to perform the task at hand within a reasonable amount of time.

    The real winner is IMHO Alessandro Furieri because he described the contest (apples and oranges, even though it was mostly JTS/GEOS vs. JTS/GEOS) most accurately.

    Posted by the person fiddling around with GRASS vector topology.

  9. and the winner is …

    JTS/GEOS because a number of different applications could successfully use JTS/GEOS to perform the task at hand within a reasonable amount of time.

    The real winner is IMHO Alessandro Furieri because he described the contest problems (apples and oranges, even though it was mostly JTS/GEOS vs. JTS/GEOS) most accurately.

    To do it in a more comparable way, the contest should IMHO be run on a (one single dedicated) virtual machine with 1 GB RAM, a 32-bit OS, and contestants would have to provide some publicly available software version, for free or for money, which would then be installed on that testing system.

    Posted by the person fiddling around with GRASS vector topology.

  10. Anonymous on said:

    I can confirm that using ArcGIS 10.1 SP1 with the 64bit Background Geoprocessing package installed can run the benchmark in 29s on a WS 2008 R2 8 cores machine (Xeon X3450 @2.67 Mhz), 8 Gb RAM.
    see https://dl.dropbox.com/u/64549871/clip.png

  11. Anonymous on said:

    I ran the whole clip on a 64 Win7 laptop (8 Gb Ram / i7 running at 3.4 Ghz) using ArcGIS 10.1 SP1 and 64 Bit Background processing in 20s.

  12. Dell Latitude, Win7 Professional (SP1) 64Bit, Intel Core i5 2,5 GHz, 4GB RAM;

    with ArcGIS 10.1 (SP1)
    Loading shape: 1 min
    Clipping: 57s

    with QGIS 1.8
    Loading shape: 3-5 min
    Clipping: 2:55 min

  13. MacBookPro, i7 2.7 GHz, 8 GB RAM
    OSX 10.8.2 Mountain Lion

    QGIS 1.8.0
    clip: 1:50 min

    GRASS 6.4.2 (from William Kyngesburye):
    clip: process killed after 45 min :(

    GRASS 7.0.svn (from Michael Barton)
    clip: process killed after 45 min (progress bar was at about 20%)

    Note: the GRASS 7.0 package used in the test is dated from 01-Nov-2012, so it is not including the latest vector improvements:
    http://trac.osgeo.org/grass/log/grass/trunk/lib/vector

    GDAL 1.9.2
    ogr2ogr -clipsrc StudyArea1MileBuffer.shp clip.shp Contours20Ft.shp
    clip: 2:16 min

  14. Based on the currently latest GRASS 7 (r53935) with fresh speed-ups: using the same old machine from 2008 as used earlier in this original blog (see above):

    - import: 26 seconds including building the topology
    - topological overlay: 4:02 min

  15. Here’s my test with PostGIS:

    4Gb RAM, Intel Core i3 540 @ 3.07 Ghz

    Ubuntu 11.10

    PostgreSQL 9.0.5 on i686-pc-linux-gnu, compiled by GCC gcc-4.5.real (Ubuntu/Linaro 4.5.2-8ubuntu4) 4.5.2, 32-bit

    POSTGIS=”2.0.1 r9979″ GEOS=”3.3.3-CAPI-1.7.4″ PROJ=”Rel. 4.8.0, 6 March 2012″ LIBXML=”2.7.8″

    Done in 3 minutes!

  16. Giovanni on said:

    For what is worth… I tested again on the same machine (i7, win 64, 16GB ram), QGIS master, ArcGIS 10 and PostGIS 2 (default configuration):

    clipping:

    2:10 on QGIS

    crash on arcgis (after more or less 1 hour)

    2:14 on PostGIS