, , , , , , ,

ArcGIS vs QGIS etc Clipping Contest Rematch revisited

Earlier this Last year, in June, Don Meltz wrote an interesting blog “ArcGIS vs QGIS Clipping Contest Rematch” where he let compete ArcGIS and Quantum GIS in a clipping contest. The benchmark contest data set in question is a 878MB ZIP file (ContourClipTest.zip with the (guessed) EPSG Code 2260 – NAD83 / New York East (ftUS)). The blog page gained a lot of comments, even from ESRI since some ArcGIS versions crashed on this test data set.

Find below the various timings compiled from the blog and the comments:

Proprietary software

Software Processing time Hardware/Software
ArcGIS 9.3 crash after 1h 9min: ERROR 999999: Error executing function. Invalid Topology [4gb file limit.] Failed to execute (Clip) unknown
ArcGIS 10.0 crash likewise unknown
ArcGIS 10.1 ESRI promise to calculate it in 34 seconds in this updated version (did anyone test?) unknown
GlobalMapper (version?) 30 mins unknown
GlobalMapper v11.02 49 sec Windows XP w/ 3.5GB RAM
Manifold 8 (64bit) 31 min Windows XP64 16 gb. RAM and 2.33 GHz

Note: The two GlobalMapper results are a bit funny, perhaps always minutes?

Free and Open Source Software

Software Processing time Hardware/Software
Quantum GIS (version?; Simple features) 4-5 min unknown
GRASS GIS 7 (topological GIS) 5 min Dell PowerEdge 2950 from 2008, Intel Xeon 2.66GHz, 8GB RAM
gvSIG to be done
PostGIS to be done

Notes: Hope volunteers will test this also on gvSIG and PostGIS (and other FOSSGIS)! Please report…

19 replies
  1. Andrew Kemp says:

    Just to add to this…Cadcorp SIS MapModeller 8.03 minutes 40.Dell Core2 laptop that at the time was being used as a web server, had chrome open and was being used for email. Will post the result later when this test is carried out on a proper desktop.

    Reply
  2. Anonymous says:

    PC: Intel core i7, 4 GB RAM; Win 7, 64-bit

    gvSIG 1.11 – 3-4 min.
    gvSIG 1.11 with SEXTANTE – 2-3 min.

    QGIS 1.8 – 4-5 min.

    MapWindow GIS 4.8.6 – 12 min.

    Reply
  3. Emilio says:

    My two cents.Intel Core2 Quad CPU Q6600 @ 2.40GHz 64 bits; 4 GB RAM; Kubuntu 12.04 Kernel 3.2.0-32-genericQGIS 1.8Loading shapefiles: 3:35 min.Clipping: 3:38 min.gvSIG 1.12Loading shapefiles: 0:20 min.Clipping: 3:27 min. (core geoprocess)OpenJUMP 1.5.2Loading shapefiles: “Out of Memory Error”

    Reply
  4. markusN says:

    Sorry for being carelss with using the word “benchmark”, of course it is a “contest” as stated in the title. It may become a benchmark when executed on the same machine with most software packages in question being installed on it.

    Reply
  5. Giovanni Manghi says:

    Just tested on my new laptop (i7, 8gb ram) with Windows 64 and qgis-dev installed with osgeo4w.It tool 0:30 to open the shape and 2:50 to do the clip. I will try test also using Linux, GRASS and other clipping tools available in the Sextante toolbox.

    Reply
  6. Markus Metz says:

    and the winner is …JTS/GEOS because a number of different applications could successfully use JTS/GEOS to perform the task at hand within a reasonable amount of time.The real winner is IMHO Alessandro Furieri because he described the contest (apples and oranges, even though it was mostly JTS/GEOS vs. JTS/GEOS) most accurately.Posted by the person fiddling around with GRASS vector topology.

    Reply
  7. Markus Metz says:

    and the winner is …JTS/GEOS because a number of different applications could successfully use JTS/GEOS to perform the task at hand within a reasonable amount of time.The real winner is IMHO Alessandro Furieri because he described the contest problems (apples and oranges, even though it was mostly JTS/GEOS vs. JTS/GEOS) most accurately.To do it in a more comparable way, the contest should IMHO be run on a (one single dedicated) virtual machine with 1 GB RAM, a 32-bit OS, and contestants would have to provide some publicly available software version, for free or for money, which would then be installed on that testing system.Posted by the person fiddling around with GRASS vector topology.

    Reply
  8. Anonymous says:

    I ran the whole clip on a 64 Win7 laptop (8 Gb Ram / i7 running at 3.4 Ghz) using ArcGIS 10.1 SP1 and 64 Bit Background processing in 20s.

    Reply
  9. matthias says:

    Dell Latitude, Win7 Professional (SP1) 64Bit, Intel Core i5 2,5 GHz, 4GB RAM;with ArcGIS 10.1 (SP1)Loading shape: 1 minClipping: 57swith QGIS 1.8Loading shape: 3-5 minClipping: 2:55 min

    Reply
  10. Carlos Grohmann says:

    MacBookPro, i7 2.7 GHz, 8 GB RAMOSX 10.8.2 Mountain LionQGIS 1.8.0clip: 1:50 minGRASS 6.4.2 (from William Kyngesburye):clip: process killed after 45 min :(GRASS 7.0.svn (from Michael Barton)clip: process killed after 45 min (progress bar was at about 20%)Note: the GRASS 7.0 package used in the test is dated from 01-Nov-2012, so it is not including the latest vector improvements:https://trac.osgeo.org/grass/log/grass/trunk/lib/vectorGDAL 1.9.2ogr2ogr -clipsrc StudyArea1MileBuffer.shp clip.shp Contours20Ft.shpclip: 2:16 min

    Reply
  11. markusN says:

    Based on the currently latest GRASS 7 (r53935) with fresh speed-ups: using the same old machine from 2008 as used earlier in this original blog (see above):- import: 26 seconds including building the topology- topological overlay: 4:02 min

    Reply
  12. Vladimir Naumov says:

    Here’s my test with PostGIS:

    4Gb RAM, Intel Core i3 540 @ 3.07 Ghz

    Ubuntu 11.10

    PostgreSQL 9.0.5 on i686-pc-linux-gnu, compiled by GCC gcc-4.5.real (Ubuntu/Linaro 4.5.2-8ubuntu4) 4.5.2, 32-bit

    POSTGIS=”2.0.1 r9979″ GEOS=”3.3.3-CAPI-1.7.4″ PROJ=”Rel. 4.8.0, 6 March 2012″ LIBXML=”2.7.8″

    Done in 3 minutes!

    Reply
  13. Giovanni says:

    For what is worth… I tested again on the same machine (i7, win 64, 16GB ram), QGIS master, ArcGIS 10 and PostGIS 2 (default configuration):

    clipping:

    2:10 on QGIS

    crash on arcgis (after more or less 1 hour)

    2:14 on PostGIS

    Reply
  14. John Lindsay says:

    I’ve just run this benchmark using the clip tool in Whitebox Geospatial Analysis Tools https://jblindsay.github.io/ghrg/Whitebox/), which is based on JTS and runs in parallel. The average of 10 iterations was 13.5178 seconds using a 3 GHz 8-core Xeon E5 Mac Pro with 64 GB of RAM.

    Benchmark Results:
    Iteration 1 = 12.04 sec.
    Iteration 2 = 11.47 sec.
    Iteration 3 = 17.305 sec.
    Iteration 4 = 11.62 sec.
    Iteration 5 = 11.489 sec.
    Iteration 6 = 11.543 sec.
    Iteration 7 = 18.781 sec.
    Iteration 8 = 11.657 sec.
    Iteration 9 = 17.611 sec.
    Iteration 10 = 11.662 sec.

    Average = 13.5178 sec.

    Reply
  15. neteler says:

    Just for the record: the huge vector dataset is still available from here:

    wget -c “https://web.archive.org/web/20130801000000/https://www.donmeltz.com/files/ContourClipTest.zip”

    Reply

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *