venerdì 7 novembre 2014

INSPIRE tools inventory

I would like to make an inventory of the tools and components aiding the INSPIRE implementation that are available as FOSS. To do so, I have created a wiki page and I encourage your contribution in filling it. 
A lot of valuable pieces of software are being developed  and they could be ideally reused without reinventing the wheel all the times. 
If you are aware of a piece of Free software that responds to this requisite, please add it to the wiki page, with a small description and a link to relevant documentation page, or contact me if you don't have an account for the OSGeo wiki and do not wish to create one.
I think that eventually of this exercise will benefit everyone interested in reusing INSPIRE components. 

Thank you for your cooperation!




giovedì 6 novembre 2014

Image segmentation in GRASS GIS

Image segmentation is the process used to detect homogeneous objects in remote sensed images. This process is useful as a first step for classification. There are several tools around, both free and proprietary, that implement different algorithms. In these days I'm testing some of them and wanted to share some impressions.
To start I have selected my favourite GIS: GRASS. I'm using GRASS 7.1. The segmentation tool that I used is implemented in a module called i.segment, that was developed by Eric Momsen of North Dakota State University, during his Google Summer of Code in 2012.
The segmentation is performed on a group of bands and not on a single band, it means that the user can select the most significant ones in respect to the features she's investigating. The bands can be grouped using the command i.group (Shapiro, 2011).
The algorithm is iterative and can be time-consuming on large images. The parameter "memory" can be used to speed up the process if the machine  has a decent memory.
The main parameter worth to play with is the threshold parameter. The larger the threshold, the more objects are merged together. If the image is big, a useful tip is to test the threshold value on a smaller region then apply it to the region at large.
Another fantastic tools comes from Pietro Zambelli and is called i.segment.hierarchical (available in add-ons). It performs hierarchical segmentation for different threshold values, using the result of the previous run as a seed for the next one. It optimizes the computational time by subdividing the image in tiles and processing them in parallel.
The results of segmentation for increasing threshold values are shown hereinafter.
Threshold = 0.01


Threshold = 0.05



Threshold = 0.1



Threshold = 0.2



lunedì 20 ottobre 2014

Call For papers Geospatial devroom @FOSDEM



Please forward!

FOSDEM is a free open source event bringing together about 5000 developers in Brussels, Belgium. The goal is to provide open source software developers and communities a place to meet at. The next edition will take place the weekend 31/1 -> 1/2/2015. This year for the first time there will be a geospatial devroom on Sunday 1/2/2015!

Geospatial technology becomes more and more part of mainstream IT. The idea is to bring together people with different backgrounds to better explain and understand the opportunities Geospatial can offer. This devroom will host topics explaining the state of the art of geospatial technology, and how it can be used amongst other projects.

The geospatial devroom is the place to talk about open, geo-related data and software and their ecosystem. This includes standards and tools, e.g. for spatial databases, and online mapping, geospatial services, used for collecting, storing, delivering, analysing, and visualizing puposes. Typical topics that will be covered are:

  • Web and desktop GIS applications
  • Interoperable geospatial web services and specifications
  • Collection of data using sensors/drones/satellites
  • Open hardware for geospatial applications
  • Geo-analytic algorithms/libraries
  • Geospatial extensions for classical databases (indexes, operations)
  • Dedicated databases

HOW TO SUBMIT YOUR TALK PROPOSAL

Are you thrilled to present your work to other open source developers? Would you like to run a discussion? Any other ideas? Please submit your proposal at the Pentabarf event planning tool at:


When submitting your talk in Pentabarf, make sure to select the 'Geospatial devroom' as  'Track'. Please specify in the notes if you prefer for your presentation a short timeslot (lightning talks ~10 minutes) or a long timeslot (20 minutes presentation + discussion).

The DEADLINE for submissions is **1st December 2014**

Should you have any questions, please do not hesitate to get in touch with the organisers of the devroom at fosdem-geospatial@gisky.be!

Johan Van de Wauw
Margherita Di Leo
Astrid Emde
Anne Ghisla
Julien Fastré
Martin Hammitzsch
Andy Petrella 
Dirk Frigne
Gael Musquet

venerdì 3 ottobre 2014

A new tool for (INSPIRE) metadata management in GRASS GIS

For those who deal with geospatial data and GRASS GIS, and need to fulfill the requirements of the INSPIRE directive for metadata, one of the most exciting outcomes of the Google Summer of Code 2014 [1] is the INSPIRE metadata support developed by Matej Krejci. This is the first step towards full metadata management through GRASS, and covers the editing part.

Purpose


The module allows to create / load / edit / export / validate xml metadata with (currently) two profiles:
  • Basic GRASS profile
  • INSPIRE profile
You can select either of them in the drop-down menu on the top left corner.

Metadata map editor


In the radio button on the top left corner, you can select either Metadata map editor or Metadata external editor. The former will modify the metadata of the maps currently present in the mapset. You can visualize them in the tree on the left hand side.





Working directory


The working directory (workdir button on the top right) is where you wish your metadata to be saved, or where you want to browse metadata from (in case you are using the Metadata external editor). The default setting is a folder called metadata stored in the current mapset.

Editing the metadata


Select the map you wish to create the metadata for. Selecting Edit will open your metadata. You will notice that some of the fields are already filled. These information, such as the date of creation, resolution and so on and so forth, are reported as detected by GRASS. You can always modify them though. 




Validation


Once your metadata is complete, you can validate it against the INSPIRE requirements, pushing the Validate button at the bottom. If you haven't filled some of the fields properly, like in this example, you will get a series of error that need to be fixed in the various tabs of the editor.





Requirements


The module wx.metadata requires GRASS 7.1 and has external dependencies:
  • OWSLib development version
git clone git://github.com/geopython/OWSLib.git
cd OWSLib && sudo python setup.py install
  • Jinja templates
git clone git://github.com/mitsuhiko/jinja2.git
cd jinja2 && sudo python setup.py install

Installation


GRASS> g.extension wx.metadata

Launch it as:

GRASS> g.gui.metadata

Notes


[1] Google Summer of Code is the program with which Google supports Open Source software, and encourages students to join and start contributing to open source communities. In fact, students selected by mentors from OS organizations, are paid a stipend to develop a project that has been identified by the community. During the "summer of code", students are constantly supported by their mentors and the OS community. OSGeo, the Open Source Geospatial Foundation, has been participating to GSoC since 2007. OSGeo serves as an umbrella organization for several OS Geospatial projects. 

venerdì 19 settembre 2014

More MOOC offers for data scientists

As promised in the previous post, I continue listing the numerous MOOC offers about Data Science, also on the basis of the feedback received.

It is worthwhile to mention that Data Analysis and Statistical Inference offers its labs through another valuable platform: DataCamp. Using this platform is like having an instructor at your side, explaining the exercise and giving you feedback. Alternatively, you can complete the exercises just using Rstudio and submitting your scripts.

On the other hand, the Data Science Specialization team has developed an innovative tool for learning R interactively: the swirl package. The idea is learning by doing, and it's fairly simple to get started with R using it. From Rstudio, all you need to do is to install the package, typing:

install.packages("swirl")

then, start swirl:

library("swirl")
swirl

And you will be in the learning environment. You have to select a course that you want to follow and the rest is really self explanatory. The learning sessions are conveniently not too long.

The University of Washington also offers a course: Introduction to Data Science, that includes the basic techniques of data science as well as databases, MapReduce, Hadoop, SQL and NoSQL. It also covers elements of statistical modelling and machine learning, as well as communication of results, and Graph analysis. The recommended (but not compulsory) textbook is "Mining of Massive Datasets". The programming assignments entail Python and SQL besides the usual R. According to the opinion of a colleague that has reviewed it, the word "Introduction" in the title is misleading, since it covers a whole lot more than an introduction.

Stanford University offers a course called Machine Learning, that is also in my watching list as I plan to review it in detail when I'll be more advanced in my studies.

Another course on Machine Learning is Learning From Data, offered by Caltech through edX. This course is currently closed but the material of the 2014 course can be consulted and studied at your own pace. It covers from the basic theory to algorithms and applications. The video lectures are also on YouTube.

Finally, a couple of more courses, both offered by Stanford, that can surely be complementary to those already mentioned, are: Statistical Learning and Introduction to Databases.

I think that with this outstanding offer, I have no excuses left for procrastination, even when I'm not at home - did you know? Coursera has an App for Android!

giovedì 18 settembre 2014

In the loop of the mooc!

Education is changing and the change is affecting our lives and the way we spend our time. I hardly believe that anyone has never heard about MOOCs, which stands for Massive Open Online Courses. Browsing among the huge offer of courses available online for free is something like discovering a new world.. or several new worlds actually.

In the variety of the courses offered by Coursera I have found utterly interesting the Data Science Specialization offered by the Johns Hopkins University, that entails 9 courses and a final Capstone Project. I appreciate this offer especially because they concentrated several essential information that if you wanted to collect otherwise you should be reading tons of books, web pages, software documentation, probably without finding immediately the connection among them.

With the recent advances in technology, trans-disciplinary concepts such as exploratory data analysis, reproducible research, regression models, machine learning, are progressively gaining importance in several fields and are shaping the "profession" of a "data scientist", a professional with a strong background in statistics as well as cutting edge expertise in technology.

So far I have successfully completed the first two courses, namely The Data Scientist's Toolbox and R programming. Since I'm a lazy person, I need to be motivated, otherwise I'll use the excuse that "I don't have time, I'll do it later". That's mainly why I enrolled in the Signature Track, in order to have deadlines, and eventually I got certificates, and shareable permanent links to course record pages, that look like this and this.

I paused in August and first week of September, and missed the beginning of the other courses, thus I'm starting again in October.

Meanwhile I've found another relevant course, partly overlapping some of the concepts of the specialization, namely Data Analysis and Statistical Inference, offered by Duke University. I'm currently enrolled in this latter, and I found several advantages: it is a lot oriented towards applied statistics and offers tons of practical examples. It also offers an excellent book (free for download, but I bought it due to the ridiculous price - and because I love paper books).

A bonus is that it doesn't require previous knowledge of statistics, which allowed me to brush up my statistics, proceeding quickly through the first weeks of course (I did the first week in one afternoon - OK, I admit it, it took me until 1.30 am).

I'm also watching another course, that is starting at the end of September: it's Mining Massive Datasets, offered by Stanford University. Yes, Big Data. I know it is probably too much in my schedule, but hey, better than mindlessly surfing Facebook in my spare time..

Next time I'll talk about more MOOC platforms and their offers..

venerdì 11 luglio 2014

Vogliamo ancora l'Italia nel registro INSPIRE

Ben 4 mesi fa chiedevamo una cosa dovuta e utile: che anche l'Italia fosse presente nel registro INSPIRE.
Non è cambiato nulla e questo incomprensibile stallo non è stato ancora superato. Ma c'è una novità e vi chiediamo di darci una mano a sostenerla e diffonderla.
La notizia di questa carenza di attuazione della normativa è arrivata a un gruppo di parlamentari (trasversale allo schieramento politico) interessato/competente sulle tematiche dell'innovazione tecnologica. L'onorevole De Lorenzis l'ha fatta sua e ha redatto questa interrogazione parlamentare, in cui chiede al Ministro dell'ambiente e della tutela del territorio e del mare "quali iniziative intenda assumere al fine di ottemperare all'obbligo di implementazione e integrazione nel portale europeo dei servizi descritti in premessa come richiesto dalla direttiva Inspire e dal relativo decreto di recepimento della stessa".
La recente campagna affinché l'Agenzia delle Entrate facesse quanto dovuto nei confronti della comunità OpenStreetMap, ha tra i tanti pregi quella di farci sentire più forti.
Noi scriveremo nei social network:
@minambienteIT @glgalletti vogliamo l'Italia nel registro INSPIRE e vogliamo una risposta http://bit.ly/italy4inspire #italy4INSPIRE
Fatelo con noi!

lunedì 10 marzo 2014

Please support the upcoming Vienna Code Sprint 2014



In the occasion of the upcoming Vienna Code Sprint 2014, the GRASS GIS Project Steering Committee decided earlier this year to  officially join this code sprint, considering it as a great opportunity  for joint activities. More than 60+ developers from the most important OSGeo project communities will be joining the event.
While the GRASS developers are donating their valuable time, the community of enthusiast users (you!) may contribute with donations,  even symbolic, that will be used to cover out-of-pocket expenses of the  participants. Companies can also decide to sponsor specific tasks!  Please don't hesitate to contact us (or Markus Neteler, neteler@osgeo.org) for further details.
As usual, all of the work done in the community sprint will be directly  contributed back to the GRASS project for the benefit of everyone who  uses it. Our scope at the sprint is to publish a first release candidate  of the stable GRASS GIS 6.4.4 version as well as a tech preview release of GRASS GIS 7.

For your convenience, here our easy-to-use Paypal button:

For our alternative bank transfer option, please contact Martin Landa (landa.martin@gmail.com)

Thanks for your support!
The GRASS Developers Team



About GRASS GIS
The Geographic Resources Analysis Support System (http://grass.osgeo.org/), commonly referred to as GRASS GIS, is an Open Source Geographic Information System providing powerful raster, vector and geospatial processing capabilities in a single integrated software suite. GRASS GIS includes tools for spatial modeling, visualization of raster and vector data, management and analysis of geospatial data, and the processing of satellite and aerial imagery. It also provides the capability to produce sophisticated presentation graphics and hardcopy maps. GRASS GIS has been translated into about twenty languages and supports a huge array of data formats. It can be used either as a stand-alone application or as backend for other software packages such as QGIS and R geostatistics. It is distributed freely under the terms of the GNU General Public License (GPL). GRASS GIS is a founding member of the Open Source Geospatial Foundation (OSGeo).


-------------

VERSIONE ITALIANA

-------------


Sostieni il Code Sprint di Vienna 2014


In occasione dell'imminente Code Sprint di Vienna 2014, all'inizio di quest'anno il Comitato Direttivo (PSC) di GRASS ha deciso di aderire ufficialmente all'evento, considerata la grande opportunita` per attivita` comuni. Piu` di 60 sviluppatori dei piu`importanti progetti OSGeo parteciperanno all'evento.
Mentre gli sviluppatori di GRASS donano il loro prezioso tempo, gli utenti appassionati (tu!) possono contribuire con donazioni, anche simboliche, che verranno utilizzate per coprire le spese vive dei partecipanti.
Le aziende possono anche decidere di sponsorizzare un compito specifico! Per saperne di piu` non esitate a contattarci (o Markus Neteler neteler@osgeo.org). 
Come sempre, tutto il lavoro svolto durante lo Sprint verra` direttamente messo a disposizione del progetto GRASS, per il beneficio dell'intera comunita` di utenti. 
L'obiettivo per lo Sprint e` quello di pubblicare la prima versione candidata stabile di GRASS GIS 6.4.4 e la versione di anteprima tecnologica di GRASS GIS 7. 
Potete utilizzare il comodo pulsante Paypal all'indirizzo: http://grass.osgeo.org/donations/
Per opzioni di pagamento alternative, potete contattare Martin Landa (landa.martin@gmail.com).

Grazie per il vostro sostegno!
Il Team degli Sviluppatori di GRASS


lunedì 10 febbraio 2014

Vogliamo anche l'Italia nel registro INSPIRE || We want Italy in the INSPIRE Registry

 

Hashtag: #italy4INSPIRE

Premessa

INSPIRE prevede che ogni Stato Membro fornisca almeno un endpoint nazionale per il discovery di metadati.
Ad oggi, la maggior parte degli Stati Membri (23 su 28) ha soddisfatto questo requisito registrando il proprio riferimento nazionale nel geoportale INSPIRE:
In particolare, come si può vedere, alcuni paesi hanno registrato più di un endpoint, come l'Austria, il Belgio e la Lettonia: è infatti possibile registrarne anche più di uno per paese.
A differenza di ciò, l’Italia non ha ancora alcun endpoint registrato per il servizio di discovery.
Per questa registrazione è necessaria una semplice comunicazione (email) del National Contact Point INSPIRE (o di qualcuno delegato dal NCP) indirizzata a EC/EEA INSPIRE Team (env-inspire@ec.europa.eu) ed per conoscenza JRC (michael.lutz@jrc.ec.europa.eu).

Domanda

Perché il servizio CSW realizzato da RNDT non è ancora stato registrato come endpoint italiano?

Dal punto di vista normativo, sia il recepimento della Direttiva INSPIRE (Dlgs. 32/2010) sia il Codice dell’Amministrazione Digitale riportano che RNDT è il riferimento nazionale in questo contesto:
Il repertorio nazionale dei dati territoriali, [...] costituisce il catalogo nazionale dei metadati relativi ai set di dati territoriali” (Dlgs. 32/2010, art.5).

Dal punto di vista tecnico-operativo i test effettuati nel luglio 2013 e gennaio 2014 dal Joint Research Centre della Commissione Europea (su richiesta dell’Agenzia per l’Italia Digitale) hanno dimostrato che il  servizio CSW del RNDT e la quasi totalità dei metadati raccolti sono perfettamente conformi a quanto previsto dai Regolamenti 1205/2008 (metadati) e 976/2009 (servizi di rete) della Commissione Europea, nonché alle relative Technical Guidelines (1.2 del 2010 per i metadati, e 3.1 del 2011 per i servizi di discovery).
In particolare il test effettuato a gennaio 2014 ha riportato 4412 metadati “passed” e 412 “passed with warnings” su un totale di 5540 metadati sottoposti ad harvesting (nel RNDT i metadati disponibili sono 6143).
Il livello di conformità rispetto a INSPIRE è quasi totale per i metadati di dataset e serie (4415 su 4462).
Questo è un risultato importante ed è da notare che risulta essere migliore rispetto ai risultati ottenuti da altri Stati Membri.
Il report completo è disponibile a questo indirizzo:

Sottolineiamo che è importante che la registrazione del servizio sia fatta al più presto perché:
1.    la disponibilità dei metadati italiani nel catalogo europeo serve a dare visibilità alle informazioni territoriali esistenti in Italia, il tutto proiettato a
            i. supportare le politiche ambientali nazionali e comunitarie
            ii.  favorire la conoscenza e la promozione del nostro territorio;
2.    l'iniziale disponibilità di metadati potrà innescare un processo virtuoso spingendo gli enti pubblici di ogni livello a conferire i metadati all'RNDT per far conoscere le attività dell’amministrazione su scala internazionale;
3.    per incentivare la realizzazione di servizi innovativi da parte di professionisti, consulenti e PMI locali da offrire agli enti locali sulla base della disponibilità di dati;
4.    per istanziare il ruolo del "nodo" Italia all'interno della rete;
5.    per dare riconoscimento e visibilità alle persone che, su scala diversa, hanno attivamente operato per la realizzazione dell'infrastruttura e dei servizi.

Conclusioni

Alla luce di queste considerazioni, esortiamo il NCP INSPIRE italiano a comunicare al più presto al JRC l'indirizzo del servizio CSW di RNDT affinché questo venga registrato come primo endpoint italiano in INSPIRE.

Firmatari (in ordine alfabetico)


  • Associazione Stati generali dell'innovazione
  • Giovanni Allegri
  • Roberto Angeletti, ExportToCanoma  blog
  • Andrea Antonello
  • Fulvio Ananasso, Stati generali dell'innovazione
  • Associazione Stati Generali dell’Innovazione
  • Associazione italiana per l’informazione geografica libera - GFOSS.it
  • Ugo Bonelli, Stati generali dell'innovazione
  • Andrea Borruso
  • Stefano Campus
  • Giovanni Ciardi
  • Piergiorgio Cipriano
  • Bruno Conte, Stati generali dell’innovazione, Social4Social
  • Simone Cortesi
  • Laura Criscuolo
  • Antonio D'Argenio, Nadir
  • Margherita Di Leo
  • Alessio Di Lorenzo
  • Gianfranco Di Pietro, Geofunction
  • Leonardo Donnaloia
  • Antonio Falciano
  • Sergio Farruggia, Stati Generali dell’Innovazione, AMFM GIS Italia
  • Daniela Ferrari
  • Maurizio Foderà, Kartoblog
  • Marco Fratoddi, Stati generali dell'innovazione
  • Antonio Fregoli, MNDAssociation
  • Pietro Blu Giandonato
  • Cesare Gerbino
  • Simone Giannecchini
  • Jacopo Grazzini
  • Nicola Guarino, ISTC-CNR
  • Giuseppe Iacono, Stati generali dell'innovazione
  • Carlo Infante, Stati generali dell'innovazione, Urban Experience
  • Andrea Latino, Stati generali dell'innovazione
  • Simone Lella
  • Walter Lorenzetti, gis3w
  • Jody Marca
  • Flavia Marzano, Stati Generali dell'Innovazione e Rete WISTER
  • Giacomo Martirano, Epsilon Italia, coordinatore progetto smeSpire
  • Stefania Morrone, Epsilon Italia
  • Beniamino Murgante, Università degli Studi della Basilicata e AMFM GIS Italia
  • Lorenzo Orlando, Stati generali dell'innovazione
  • Alessandro Oggioni
  • Mariella Pappalepore, Planetek Italia
  • Stefano Parodi, GeoWebLog
  • Lorenzo Perone
  • Emma Pietrafesa, Stati generali dell’innovazione (Rete WISTER)
  • Renzo Provedel, Stati generali dell’innovazione, SOSLOG
  • Angelo Quaglia
  • Morena Ragone, Stati generali dell'innovazione
  • Paolo Russo, Stati generali dell'innovazione
  • Alessandro Sarretta
  • Monica Sebillo, AMFM GIS Italia
  • Gian Bartolomeo Siletto
  • Claudia Spinnato, Consorzio TICONZERO
  • Lorenzino Vaccari, Provincia Autonoma Trento
  • Franco Vico, AMFM GIS Italia
  • Fabio Vinci, Epsilon Italia
  • Massimo Zotti

-------------- %< -------------- %< -------------- %< --------------
ENGLISH VERSION
Petition to the Italian NCP for the RNDT registration in INSPIRE

Premise
According to INSPIRE each Member State must provide at least one endpoint for the discovery of national metadata.
To date, the majority of Member States (23 out of 28) met this requirement by registering its national endpoint in the INSPIRE geo-portal :
In particular it’s worth noting that some Member States have recorded more than one endpoint, e.g. Austria, Belgium and Latvia : it is also possible to register multiple national endpoints.
In spite of this, Italy has not yet registered its endpoint for the discovery service.
In order to get this registration, a simple communication is needed (e-mail), sent by the INSPIRE National Contact Point addressed to EC/EEA INSPIRE Team (env-inspire@ec.europa.eu) and the JRC (michael.lutz@jrc.ec.europa.eu).

Question
Why the CSW service exposed ​​by the RNDT has not yet been registered as one of the Italian endpoints?
From the regulatory point of view , both the transposition of the INSPIRE Directive (Legislative Decree 32/2010 ) and the Codice dell’Amministrazione Digitale report that RNDT is the national reference in this context :
"The national repertoire of spatial data, [... ] is the national catalog of metadata for spatial data sets" (Legislative Decree 32/2010 , Article 5).

From the technical and operational point of view, otherwise, the tests carried out in July 2013 and January 2014 by the Joint Research Centre of the European Commission ( upon request of the Agenzia per l’Italia Digitale) have shown that the CSW service and almost all of the metadata harvested are fully compliant to the provisions of Regulations 1205/2008 (metadata) and 976/2009 (network services) of the European Commission, as well as the related Technical Guidelines ( 1.2 of 2010 for metadata , and 3.1 of 2011 for discovery services) .
Notably the test performed in January 2014 reported 4412 metadata "passed" and 412 "passed with warnings " out of a total of 5540 " harvested "metadata  (the total available metadata in the RNDT are 6143) .
The level of compliance to INSPIRE is almost complete for the metadata related to data (4415 of 4462) .
This is an important result and it is noteworthy better than the results obtained by other Member States.
The full report is available at this address :

We emphasize that it is important that the registration of the service is done as soon as possible because :
  1. the availability of Italian metadata in the European catalog is needed to give visibility to the spatial information existing in Italy , in order to
i. support national and EU environmental policies;
ii. improve knowledge of and encourage investment in our country;
  1. the initial availability of metadata can trigger a virtuous cycle by pushing government entities at every level to provide new metadata to RNDT,  to promote their activities at the international level;
  2. to encourage the creation of innovative services by professionals, consultants and local SMEs, based on the availability of data, to the benefit of local authorities;
  3. to instantiate the role of the Italian " node " within the European network ;
  4. to give visibility and recognition to people who, at different scales, have actively worked for the implementation of infrastructures and services .

Conclusions
On the basis of the aforementioned considerations, we urge the INSPIRE National Contact Point to provide as soon as possible the URL of the RNDT CSW, in order to have the first italian endpoint registered in INSPIRE.

Signature (in alphabetic order)
  • (see list above)