Unable to ingest JSON messages from Kafka

[Protocol-X]

I get the following error, but i have checked the message schema multiple times and it is correct.

[ERROR] PipelineErrorServiceUtil - Not initialized data retrieval from messaging service init method needs to be called or messaging and exchange object should be pre instantiated and class member before calling init for tenant: 2, topplogy: SHTEST_TestPipeline_Inspect_1519763949260, component: Kafka_144629
ApplicationException [message=Message parsing failed. Could not de-serialize data, probable reason is that message has wrong schema., throwable=null, errorCode=null, jsonData={"data":"{\"pipelineName\":\"SHTEST_TestPipeline_Inspect_1519763949260\",\"tenantId\":\"2\",\"componentName\":\"Kafka_144629\"}","errorMessage":"Message parsing failed. Could not de-serialize data, probable reason is that message has wrong schema."}]

[sameerb]

Hello!
Can you upload the sample JSON & the pipeline on this thread?
Exporting the pipeline is easy - click the "data pipeline" link; then click the download button on the pipeline tile.
The pipeline logs are under /logs/pipeline-logs/
The complete stacktrace should be available there.

[Protocol-X]

Thanks for the reply sameerb. I was able to get it working, one key was missing. I assumed the application was smart enough to ignore and or self discover key value pairs bot declared. Hopefully in the future they will add generic self discovery as it is tedious to list every key. I stillhavent been able to get it to write to Kafka though. No errors when I execute the pipeline it just keeps toggling between starting and active. There doesn't appear to be any visual logging system to see what is going on. I guess I need to dig up the logs in the location you provided. Are you part of the team or a user?

[sameerb]

I work with the product team.
JSON message schema is auto-created by uploading sample JSON; instead of keying in every key in the message.
Missing keys in the metadata will cause the kafka channel to ignore those keys in the message.
Meta discovery is available in technical preview but it is not available in VSS.
There is a visual logging monitoring framework which is part of StreamAnalytix enterprise deployments.