Good tool for analyzing a request log for an OGC service like a WMS or WFS service

Question

Background

There are many tools to analyze logfiles from servers like Apache HTTP server, but I need one to analyze requests to geospatial services, WMS and WFS

What is special about a request to a OGC service like WMS or WFS? The protocol includes use of geographical request parameters (see e.g. parameters for WMS GetMap request), like a bounding box, or geographical filter, i.e. OGC filter.

Examples of questions I want answered include:

• What is the most common bounding box being requested, boundingbox = (minX,minY,maxX,maxY)
• Do bounding boxes in requests form a "grid", and if so, which grids?

I could import all the log-lines into a spatial database like PostGIS, and then the problem reduces to writing SQL, which is also ok. But it is not trivial to import that many rows on the system I currently have, and then I still need good formulations in SQL.

So, Ideally I'd like a tool that eats log lines and understands OGC WMS, WCS, WFS, CSW etc.

The problem of Googling this question is the proximity to web log analizers, and WMS = windows media server, and the relative obscurity of the area (geospatial). Maybe the answer you provide is a good google search formulation?

Question

Does a tool exist to analyze OGC request logs, to discover things like a recurring spatial pattern?

Answer 1

We have done such a tool for the French Geological Survey. It analyzes apache logs and retrieves services, layers, srs, user-agents, servers from ogc requests and build charts and tables about it. It can also record bbox of each request in order to display a heat map of service usage. It could be easily brought up to OpenSource, so don't hesitate to contact me for further discussion.

Best regards

Answer 2

There's a dump of some initial code I did awhile back at https://github.com/tomkralidis/pyowslog. The general idea is implemented, but would need additional work for bbox analysis.