Filter on day of the week and aggregate on hour

255 views Asked by At

For my master, I have a programming course in which I have to work with Elasticsearch. I have a dataset with a few million Ubereat orders. Every record consists of the dateTime, restaurant, pickup point and dropoff point. I had to distinguish different days and report the number of orders on that specific day. So, for example, I managed to find the number of orders on Christmas, per hour.

I also managed to find the total amount of orders on Monday, Tuesday, Wednesday etc, for the entire year. Now, I have to combine these two. I need to know how much orders were placed for the entire year on Monday, Tuesday, Wednesday etc. between 9:00 and 10:00, 10:00 and 11:00, 11:00 and 12:00 etc. I have included my attempt. However, I do get an error:

RequestError(400, 'parsing_exception', '[bool] malformed query, expected [END_OBJECT] but found [FIELD_NAME]')

Does someone know how to combine this query and aggregation?

search_body = {
 'size': 0,
 'query': {
    'range':{
        'dateTime':{
            'gte': "2018-12-25 00:00:00",
            'lte': "2018-12-26 00:00:00"
        }
    }
},
    "aggs": {
    "orders_per_date": {
      "date_histogram": {
        "field": "dateTime",
        "calendar_interval": "hour"
      }
    }
 }
}

search_body = {
      "size": 0,
      "aggs": {
            "total_orders_per_day_of_week": {
                  "terms": {
                        "script": {
                              "lang": "painless",
                              "source": "doc['dateTime'].value.dayOfWeek"
                        }
                  }


search_body = {
  "size": 0,
  "query": {
    "bool": {
      "filter": {
        "script": {
          "script": {
            "source": "doc['dateTime'].value.dayOfWeek",
            "lang": "painless"
          }
        }
      }
     },
      "aggs": {
          "total_order_per_hour_per_day": {
            "date_histogram": {
              "field": "dateTime",
              "calendar_interval": "hour"
             }
            }
        }
    }
}

[2020-10-10T15:06:57,628][INFO ][o.e.n.Node               ] [DESKTOP-1OP6K1E] version[7.9.2], pid[10380], build[default/zip/d34da0ea4a966c4e49417f2da2f244e3e97b4e6e/2020-09-23T00:45:33.626720Z], OS[Windows 10/10.0/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/15/15+36]
[2020-10-10T15:06:57,644][INFO ][o.e.n.Node               ] [DESKTOP-1OP6K1E] JVM home [C:\Users\leand\Desktop\elasticsearch-7.9.2\jdk]
[2020-10-10T15:06:57,644][INFO ][o.e.n.Node               ] [DESKTOP-1OP6K1E] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=C:\Users\leand\AppData\Local\Temp\elasticsearch, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=536870912, -Delasticsearch, -Des.path.home=C:\Users\leand\Desktop\elasticsearch-7.9.2, -Des.path.conf=C:\Users\leand\Desktop\elasticsearch-7.9.2\config, -Des.distribution.flavor=default, -Des.distribution.type=zip, -Des.bundled_jdk=true]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [aggs-matrix-stats]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [analysis-common]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [constant-keyword]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [flattened]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [frozen-indices]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [ingest-common]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [ingest-geoip]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [ingest-user-agent]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [kibana]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [lang-expression]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [lang-mustache]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [lang-painless]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [mapper-extras]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [parent-join]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [percolator]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [rank-eval]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [reindex]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [repository-url]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [search-business-rules]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [searchable-snapshots]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [spatial]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [tasks]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [transform]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [transport-netty4]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [vectors]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [wildcard]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-analytics]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-async]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-async-search]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-autoscaling]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-ccr]
[2020-10-10T15:07:03,707][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-core]
[2020-10-10T15:07:03,723][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-data-streams]
[2020-10-10T15:07:03,723][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-deprecation]
[2020-10-10T15:07:03,723][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-enrich]
[2020-10-10T15:07:03,723][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-eql]
[2020-10-10T15:07:03,723][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-graph]
[2020-10-10T15:07:03,723][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-identity-provider]
[2020-10-10T15:07:03,723][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-ilm]
[2020-10-10T15:07:03,723][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-logstash]
[2020-10-10T15:07:03,723][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-ml]
[2020-10-10T15:07:03,723][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-monitoring]
[2020-10-10T15:07:03,723][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-ql]
[2020-10-10T15:07:03,723][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-rollup]
[2020-10-10T15:07:03,723][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-security]
[2020-10-10T15:07:03,723][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-sql]
[2020-10-10T15:07:03,723][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-stack]
[2020-10-10T15:07:03,723][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-voting-only-node]
[2020-10-10T15:07:03,723][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-watcher]
[2020-10-10T15:07:03,723][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] no plugins loaded
[2020-10-10T15:07:04,046][INFO ][o.e.e.NodeEnvironment    ] [DESKTOP-1OP6K1E] using [1] data paths, mounts [[(C:)]], net usable_space [128.5gb], net total_space [232.4gb], types [NTFS]
[2020-10-10T15:07:04,047][INFO ][o.e.e.NodeEnvironment    ] [DESKTOP-1OP6K1E] heap size [1gb], compressed ordinary object pointers [true]
[2020-10-10T15:07:04,295][INFO ][o.e.n.Node               ] [DESKTOP-1OP6K1E] node name [DESKTOP-1OP6K1E], node ID [m3Cz932jSqOozhQ0ua9OVg], cluster name [elasticsearch]
[2020-10-10T15:07:07,389][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [DESKTOP-1OP6K1E] [controller/1928] [Main.cc@114] controller (64 bit): Version 7.9.2 (Build 6a60f0cf2dd5a5) Copyright (c) 2020 Elasticsearch BV
[2020-10-10T15:07:07,907][INFO ][o.e.x.s.a.s.FileRolesStore] [DESKTOP-1OP6K1E] parsed [0] roles from file [C:\Users\leand\Desktop\elasticsearch-7.9.2\config\roles.yml]
[2020-10-10T15:07:08,789][INFO ][o.e.t.NettyAllocator     ] [DESKTOP-1OP6K1E] creating NettyAllocator with the following configs: [name=unpooled, factors={es.unsafe.use_unpooled_allocator=false, g1gc_enabled=true, g1gc_region_size=1mb, heap_size=1gb}]
[2020-10-10T15:07:08,850][INFO ][o.e.d.DiscoveryModule    ] [DESKTOP-1OP6K1E] using discovery type [zen] and seed hosts providers [settings]
[2020-10-10T15:07:09,244][WARN ][o.e.g.DanglingIndicesState] [DESKTOP-1OP6K1E] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
[2020-10-10T15:07:09,658][INFO ][o.e.n.Node               ] [DESKTOP-1OP6K1E] initialized
[2020-10-10T15:07:09,658][INFO ][o.e.n.Node               ] [DESKTOP-1OP6K1E] starting ...
[2020-10-10T15:07:09,824][INFO ][o.e.t.TransportService   ] [DESKTOP-1OP6K1E] publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}, {[::1]:9300}
[2020-10-10T15:07:10,226][WARN ][o.e.b.BootstrapChecks    ] [DESKTOP-1OP6K1E] the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
[2020-10-10T15:07:10,228][INFO ][o.e.c.c.Coordinator      ] [DESKTOP-1OP6K1E] cluster UUID [O_T444u1TtOptSi2tnYMHA]
[2020-10-10T15:07:10,239][INFO ][o.e.c.c.ClusterBootstrapService] [DESKTOP-1OP6K1E] no discovery configuration found, will perform best-effort cluster bootstrapping after [3s] unless existing master is discovered
[2020-10-10T15:07:10,319][INFO ][o.e.c.s.MasterService    ] [DESKTOP-1OP6K1E] elected-as-master ([1] nodes joined)[{DESKTOP-1OP6K1E}{m3Cz932jSqOozhQ0ua9OVg}{JqCVbGVRQe69M_eiFTrlaw}{127.0.0.1}{127.0.0.1:9300}{dilmrt}{ml.machine_memory=17108164608, xpack.installed=true, transform.node=true, ml.max_open_jobs=20} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 13, version: 327, delta: master node changed {previous [], current [{DESKTOP-1OP6K1E}{m3Cz932jSqOozhQ0ua9OVg}{JqCVbGVRQe69M_eiFTrlaw}{127.0.0.1}{127.0.0.1:9300}{dilmrt}{ml.machine_memory=17108164608, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}]}
[2020-10-10T15:07:10,449][INFO ][o.e.c.s.ClusterApplierService] [DESKTOP-1OP6K1E] master node changed {previous [], current [{DESKTOP-1OP6K1E}{m3Cz932jSqOozhQ0ua9OVg}{JqCVbGVRQe69M_eiFTrlaw}{127.0.0.1}{127.0.0.1:9300}{dilmrt}{ml.machine_memory=17108164608, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}]}, term: 13, version: 327, reason: Publication{term=13, version=327}
[2020-10-10T15:07:10,513][INFO ][o.e.h.AbstractHttpServerTransport] [DESKTOP-1OP6K1E] publish_address {127.0.0.1:9200}, bound_addresses {127.0.0.1:9200}, {[::1]:9200}
[2020-10-10T15:07:10,515][INFO ][o.e.n.Node               ] [DESKTOP-1OP6K1E] started
[2020-10-10T15:07:10,787][INFO ][o.e.l.LicenseService     ] [DESKTOP-1OP6K1E] license [f3871a4a-f01a-4943-bfa6-86c1a1c10ff6] mode [basic] - valid
[2020-10-10T15:07:10,789][INFO ][o.e.x.s.s.SecurityStatusChangeListener] [DESKTOP-1OP6K1E] Active license is now [BASIC]; Security is disabled
[2020-10-10T15:07:10,799][INFO ][o.e.g.GatewayService     ] [DESKTOP-1OP6K1E] recovered [7] indices into cluster_state
[2020-10-10T15:07:13,147][INFO ][o.e.c.r.a.AllocationService] [DESKTOP-1OP6K1E] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[reken][0]]]).
[2020-10-10T16:18:00,511][INFO ][o.e.x.m.p.NativeController] [DESKTOP-1OP6K1E] Native controller process has stopped - no new native processes can be started
[2020-10-10T16:18:00,511][INFO ][o.e.b.Bootstrap          ] [DESKTOP-1OP6K1E] running graceful exit on windows
[2020-10-10T16:18:00,511][INFO ][o.e.n.Node               ] [DESKTOP-1OP6K1E] stopping ...
[2020-10-10T16:18:00,511][INFO ][o.e.x.w.WatcherService   ] [DESKTOP-1OP6K1E] stopping watch service, reason [shutdown initiated]
[2020-10-10T16:18:00,511][INFO ][o.e.x.w.WatcherLifeCycleService] [DESKTOP-1OP6K1E] watcher has stopped and shutdown
[2020-10-10T16:18:00,662][INFO ][o.e.n.Node               ] [DESKTOP-1OP6K1E] stopped
[2020-10-10T16:18:00,662][INFO ][o.e.n.Node               ] [DESKTOP-1OP6K1E] closing ...
[2020-10-10T16:18:00,676][INFO ][o.e.n.Node               ] [DESKTOP-1OP6K1E] closed
[2020-10-10T16:53:40,977][INFO ][o.e.n.Node               ] [DESKTOP-1OP6K1E] version[7.9.2], pid[4416], build[default/zip/d34da0ea4a966c4e49417f2da2f244e3e97b4e6e/2020-09-23T00:45:33.626720Z], OS[Windows 10/10.0/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/15/15+36]
[2020-10-10T16:53:40,977][INFO ][o.e.n.Node               ] [DESKTOP-1OP6K1E] JVM home [C:\Users\leand\Desktop\elasticsearch-7.9.2\jdk]
[2020-10-10T16:53:40,977][INFO ][o.e.n.Node               ] [DESKTOP-1OP6K1E] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=C:\Users\leand\AppData\Local\Temp\elasticsearch, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=536870912, -Delasticsearch, -Des.path.home=C:\Users\leand\Desktop\elasticsearch-7.9.2, -Des.path.conf=C:\Users\leand\Desktop\elasticsearch-7.9.2\config, -Des.distribution.flavor=default, -Des.distribution.type=zip, -Des.bundled_jdk=true]
[2020-10-10T16:53:43,025][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [aggs-matrix-stats]
[2020-10-10T16:53:43,025][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [analysis-common]
[2020-10-10T16:53:43,025][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [constant-keyword]
[2020-10-10T16:53:43,025][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [flattened]
[2020-10-10T16:53:43,026][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [frozen-indices]
[2020-10-10T16:53:43,026][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [ingest-common]
[2020-10-10T16:53:43,026][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [ingest-geoip]
[2020-10-10T16:53:43,026][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [ingest-user-agent]
[2020-10-10T16:53:43,026][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [kibana]
[2020-10-10T16:53:43,027][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [lang-expression]
[2020-10-10T16:53:43,027][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [lang-mustache]
[2020-10-10T16:53:43,027][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [lang-painless]
[2020-10-10T16:53:43,027][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [mapper-extras]
[2020-10-10T16:53:43,027][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [parent-join]
[2020-10-10T16:53:43,028][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [percolator]
[2020-10-10T16:53:43,028][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [rank-eval]
[2020-10-10T16:53:43,028][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [reindex]
[2020-10-10T16:53:43,029][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [repository-url]
[2020-10-10T16:53:43,029][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [search-business-rules]
[2020-10-10T16:53:43,030][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [searchable-snapshots]
[2020-10-10T16:53:43,030][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [spatial]
[2020-10-10T16:53:43,031][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [tasks]
[2020-10-10T16:53:43,031][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [transform]
[2020-10-10T16:53:43,031][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [transport-netty4]
[2020-10-10T16:53:43,032][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [vectors]
[2020-10-10T16:53:43,032][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [wildcard]
[2020-10-10T16:53:43,032][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-analytics]
[2020-10-10T16:53:43,033][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-async]
[2020-10-10T16:53:43,033][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-async-search]
[2020-10-10T16:53:43,034][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-autoscaling]
[2020-10-10T16:53:43,036][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-ccr]
[2020-10-10T16:53:43,036][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-core]
[2020-10-10T16:53:43,037][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-data-streams]
[2020-10-10T16:53:43,037][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-deprecation]
[2020-10-10T16:53:43,038][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-enrich]
[2020-10-10T16:53:43,038][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-eql]
[2020-10-10T16:53:43,038][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-graph]
[2020-10-10T16:53:43,039][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-identity-provider]
[2020-10-10T16:53:43,039][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-ilm]
[2020-10-10T16:53:43,039][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-logstash]
[2020-10-10T16:53:43,040][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-ml]
[2020-10-10T16:53:43,040][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-monitoring]
[2020-10-10T16:53:43,040][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-ql]
[2020-10-10T16:53:43,041][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-rollup]
[2020-10-10T16:53:43,041][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-security]
[2020-10-10T16:53:43,041][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-sql]
[2020-10-10T16:53:43,042][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-stack]
[2020-10-10T16:53:43,042][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-voting-only-node]
[2020-10-10T16:53:43,043][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] loaded module [x-pack-watcher]
[2020-10-10T16:53:43,043][INFO ][o.e.p.PluginsService     ] [DESKTOP-1OP6K1E] no plugins loaded
[2020-10-10T16:53:43,312][INFO ][o.e.e.NodeEnvironment    ] [DESKTOP-1OP6K1E] using [1] data paths, mounts [[(C:)]], net usable_space [128.5gb], net total_space [232.4gb], types [NTFS]
[2020-10-10T16:53:43,312][INFO ][o.e.e.NodeEnvironment    ] [DESKTOP-1OP6K1E] heap size [1gb], compressed ordinary object pointers [true]
[2020-10-10T16:53:43,455][INFO ][o.e.n.Node               ] [DESKTOP-1OP6K1E] node name [DESKTOP-1OP6K1E], node ID [m3Cz932jSqOozhQ0ua9OVg], cluster name [elasticsearch]
[2020-10-10T16:53:46,137][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [DESKTOP-1OP6K1E] [controller/516] [Main.cc@114] controller (64 bit): Version 7.9.2 (Build 6a60f0cf2dd5a5) Copyright (c) 2020 Elasticsearch BV
[2020-10-10T16:53:46,617][INFO ][o.e.x.s.a.s.FileRolesStore] [DESKTOP-1OP6K1E] parsed [0] roles from file [C:\Users\leand\Desktop\elasticsearch-7.9.2\config\roles.yml]
[2020-10-10T16:53:47,342][INFO ][o.e.t.NettyAllocator     ] [DESKTOP-1OP6K1E] creating NettyAllocator with the following configs: [name=unpooled, factors={es.unsafe.use_unpooled_allocator=false, g1gc_enabled=true, g1gc_region_size=1mb, heap_size=1gb}]
[2020-10-10T16:53:47,389][INFO ][o.e.d.DiscoveryModule    ] [DESKTOP-1OP6K1E] using discovery type [zen] and seed hosts providers [settings]
[2020-10-10T16:53:47,717][WARN ][o.e.g.DanglingIndicesState] [DESKTOP-1OP6K1E] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
[2020-10-10T16:53:48,016][INFO ][o.e.n.Node               ] [DESKTOP-1OP6K1E] initialized
[2020-10-10T16:53:48,016][INFO ][o.e.n.Node               ] [DESKTOP-1OP6K1E] starting ...
[2020-10-10T16:53:48,138][INFO ][o.e.t.TransportService   ] [DESKTOP-1OP6K1E] publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}, {[::1]:9300}
[2020-10-10T16:53:48,425][WARN ][o.e.b.BootstrapChecks    ] [DESKTOP-1OP6K1E] the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
[2020-10-10T16:53:48,425][INFO ][o.e.c.c.Coordinator      ] [DESKTOP-1OP6K1E] cluster UUID [O_T444u1TtOptSi2tnYMHA]
[2020-10-10T16:53:48,438][INFO ][o.e.c.c.ClusterBootstrapService] [DESKTOP-1OP6K1E] no discovery configuration found, will perform best-effort cluster bootstrapping after [3s] unless existing master is discovered
[2020-10-10T16:53:48,506][INFO ][o.e.c.s.MasterService    ] [DESKTOP-1OP6K1E] elected-as-master ([1] nodes joined)[{DESKTOP-1OP6K1E}{m3Cz932jSqOozhQ0ua9OVg}{aSryhp_tSaaO7aY4dTPcvg}{127.0.0.1}{127.0.0.1:9300}{dilmrt}{ml.machine_memory=17108164608, xpack.installed=true, transform.node=true, ml.max_open_jobs=20} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 14, version: 345, delta: master node changed {previous [], current [{DESKTOP-1OP6K1E}{m3Cz932jSqOozhQ0ua9OVg}{aSryhp_tSaaO7aY4dTPcvg}{127.0.0.1}{127.0.0.1:9300}{dilmrt}{ml.machine_memory=17108164608, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}]}
[2020-10-10T16:53:48,616][INFO ][o.e.c.s.ClusterApplierService] [DESKTOP-1OP6K1E] master node changed {previous [], current [{DESKTOP-1OP6K1E}{m3Cz932jSqOozhQ0ua9OVg}{aSryhp_tSaaO7aY4dTPcvg}{127.0.0.1}{127.0.0.1:9300}{dilmrt}{ml.machine_memory=17108164608, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}]}, term: 14, version: 345, reason: Publication{term=14, version=345}
[2020-10-10T16:53:48,667][INFO ][o.e.h.AbstractHttpServerTransport] [DESKTOP-1OP6K1E] publish_address {127.0.0.1:9200}, bound_addresses {127.0.0.1:9200}, {[::1]:9200}
[2020-10-10T16:53:48,683][INFO ][o.e.n.Node               ] [DESKTOP-1OP6K1E] started
[2020-10-10T16:53:48,854][INFO ][o.e.l.LicenseService     ] [DESKTOP-1OP6K1E] license [f3871a4a-f01a-4943-bfa6-86c1a1c10ff6] mode [basic] - valid
[2020-10-10T16:53:48,870][INFO ][o.e.x.s.s.SecurityStatusChangeListener] [DESKTOP-1OP6K1E] Active license is now [BASIC]; Security is disabled
[2020-10-10T16:53:48,870][INFO ][o.e.g.GatewayService     ] [DESKTOP-1OP6K1E] recovered [7] indices into cluster_state
[2020-10-10T16:53:50,353][INFO ][o.e.c.r.a.AllocationService] [DESKTOP-1OP6K1E] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[persons][0]]]).
[2020-10-10T16:56:35,690][INFO ][o.e.x.m.p.NativeController] [DESKTOP-1OP6K1E] Native controller process has stopped - no new native processes can be started
[2020-10-10T16:56:35,705][INFO ][o.e.b.Bootstrap          ] [DESKTOP-1OP6K1E] running graceful exit on windows
[2020-10-10T16:56:35,705][INFO ][o.e.n.Node               ] [DESKTOP-1OP6K1E] stopping ...
[2020-10-10T16:56:35,705][INFO ][o.e.x.w.WatcherService   ] [DESKTOP-1OP6K1E] stopping watch service, reason [shutdown initiated]
[2020-10-10T16:56:35,705][INFO ][o.e.x.w.WatcherLifeCycleService] [DESKTOP-1OP6K1E] watcher has stopped and shutdown
[2020-10-10T16:56:36,058][INFO ][o.e.n.Node               ] [DESKTOP-1OP6K1E] stopped
[2020-10-10T16:56:36,058][INFO ][o.e.n.Node               ] [DESKTOP-1OP6K1E] closing ...
[2020-10-10T16:56:36,073][INFO ][o.e.n.Node               ] [DESKTOP-1OP6K1E] closed
1

There are 1 answers

4
ESCoder On

RequestError(400, 'parsing_exception', '[bool] malformed query, expected [END_OBJECT] but found [FIELD_NAME]')

The error clearly indicates that the query is not formed correctly. The query block needs to be closed, and then the aggregation block is there.

Try out this search query:

{
  "size": 0,
  "query": {
    "bool": {
      "filter": {
        "script": {
          "script": {
            "source": "doc['dateTime'].value.dayOfWeek",
            "lang": "painless"
          }
        }
      }
    }
  },                     <-- note the curly brackets
  "aggs": {
    "total_order_per_hour_per_day": {
      "date_histogram": {
        "field": "dateTime",
        "calendar_interval": "hour"
      }
    }
  }
}