Database Without a database? [Advice needed]

Question

For a school project we need to store a lot of weather data. 8000 rows per second are supposed to be inserted. This data also needs to be queried fast, with filters, groups, sorting and limiting.

But we are not allowed to use an existing database management system like mySql or Mongo.

Currently we are thinking about creating a new file every second with 8000 lines each. Only reading the data back from all those files to generate a top 10 list would not be fast. So fast querying is the main problem.

Any recommendations are welcome.

EDIT:

We need to store 8000 rows per second containing the following values from from xml files we receive. We already can parse and store 8000 of these xml files per second in flat files(CSV style). But we are not able to do any fast querying. An example query would be: Get the highest 100 temperatures from the latest 8 million rows.

<MEASUREMENT>
    <STN>123456</STN>
    <DATE>2009-09-13</DATE>
    <TIME>15:59:46</TIME>
    <TEMP>-60.1</TEMP>
    <DEWP>-58.1</DEWP>
    <STP>1034.5</STP>
    <SLP>1007.6</SLP>
    <VISIB>123.7</VISIB>
    <WDSP>10.8</WDSP>
    <PRCP>11.28</PRCP>
    <SNDP>11.1</SNDP>
    <FRSHTT>010101</FRSHTT>
    <CLDC>87.4</CLDC>
    <WNDDIR>342</WNDDIR>
</MEASUREMENT>

Answer 1

Try out clickhouse (https://clickhouse.yandex/) :) It's kinda overkill, but it will work very fast!

edit: You are not allowed use ANY database? If so, than use flat files.

Answer 2

How about using SQLite? You can embed it in your Project and tweak for performance. Also I would check here: https://en.wikipedia.org/wiki/Embedded_database