Data is not visible in Azure data explorer after ingestion from Event hub

426 views Asked by At

I want to send the data from my code to Azure Event hub and finally to Azure Data Explorer. I sent the json data to event hub using typescript with the help of following code snippet:

async function sendToEventHub(data: any) {
  const producerClient = new EventHubProducerClient(connectionString, eventHubName);
  let batch = await producerClient.createBatch();
  batch.tryAdd({body: data});
  await producerClient.sendBatch(batch);
  await producerClient.close();
};

const data = {
  "runId": "abc",
  "startTime": "1234",
  "endTime": "1234",
  "elapsedTime": "1234"
};

sendToEventHub(data).catch((err) => {
  console.log("Error occurred: ", err);
});

I can see the incoming messages in event hub monitors. Then I created Azure Data explorer cluster and database. I ingested the data into this database and created new table 'test'. But when I ran the query to see the contents of table, it is coming as empty. You can see the snapshot here, Also see this.

I can interpret that the data is coming to data explorer but not able to show as the table schema(column names and types) was created with the help of data itself.

What am I missing here? I could see the data till the last step in the preview of ingestion of the data here.

2

There are 2 answers

0
Ikhtesam Afrin On BEST ANSWER

I have used your code but it worked for me. The only modification I did is providing the data in the correct JSON format

{ "name": "Afreen", "age": 26 }

Code:

import { EventHubProducerClient, EventData } from  "@azure/event-hubs";
const  connectionString  =  "Your Event Hub connection string";
const  eventHubName  =  "Event Hub name";
async  function  sendToEventHub(data:  any) {
const  producerClient  =  new  EventHubProducerClient(connectionString, eventHubName);
let  batch  =  await  producerClient.createBatch();
batch.tryAdd({body:  data});
await  producerClient.sendBatch(batch);
await  producerClient.close();
};
const  data  = {
"runId":  "xyz",
"startTime":  1234,
"endTime":  1234,
"elapsedTime":  1234
};
sendToEventHub(data).catch((err) => {
console.log("Error occurred: ", err);
});

Azure Data Explorer:

enter image description here

enter image description here

0
Vladik Branevich On

Judging by the images you attached, your data fails to be parsed according to the format you specified. So you should check:

  1. How is your data connection defined? What is the data format?
  2. Check if anything shows with .show ingestion failures