Community Discussion

Column family 'gepoint' or 'geopoint'

hieuiph 2017-03-22 10:28:28 UTC #1

Hi,

I am trying do the Data Enrichment and I'm a little bit confused about the name of column family 'gepoint' or 'geopoint' ?

in the getting start, you mentioned that:

"ip2geo" table in hbase maps ip to latitude & longitude values. These values are stored in ‘geocoordinates’ column with column family ‘gepoint’.

but in another topic question I see that:

hbase(main):001:0> scan 'ip2geo'

ROW COLUMN+CELL

11.159.5.13 column=geopoint:geocoordinates, timestamp=1486217298919, value=39.96678,-83.01269

12.113.211.20 column=geopoint:geocoordinates, timestamp=1486217296472, value=34.052212,-118.24356

4.16.86.45 column=geopoint:geocoordinates, timestamp=1486217297216, value=37.77428,-122.41983

4.2.171.91 column=geopoint:geocoordinates, timestamp=1486217296918, value=40.44745,-79.99082

anupam.jain 2017-03-22 10:50:46 UTC #2

Hi,

Its geopoint. A sample mvel to get geocoordinates is

$lookupHBase('ip2geo',@{NetworkConnectionDetails.ip}, 'geopoint')[0].get('geocoordinates')

hieuiph 2017-03-22 10:55:13 UTC #3

Hi,

thanks. I tried with same your mvel but after inspect pipeline, when I see in the output of enricher, the geopoint filed is null value. Is that normal ?

anupam.jain 2017-03-22 11:43:30 UTC #4

I tried the inspect, verified on enricher output, the geopoint is not null as attached screen shot.

Inspect is just way to identify the input output flow, so data is not actually saving to any emitters. For that pipeline needs to be run.

Also found that your have uploaded the sample data file for the first time and ran the inspect, due to this data is deleted. So kindly re-upload the data and then save the pipeline at least once. After that you can verify by running or inspect

hieuiph 2017-03-23 02:01:35 UTC #6

Hi,

thank you. I ran it successfully.

hieuiph 2017-03-23 02:27:12 UTC #7

one more question about ElasticSearch emitter: When I inspect pipeline, the output of Elastic Search is 0

My ES configuration:

hieuiph 2017-03-23 02:28:09 UTC #8

ignore my question. I understood.

kalyanischakravarthi 2017-03-23 04:02:09 UTC #9

By the way i have a question to you. How could you manage to connect o hbase shell

rknimmakayala 2017-03-23 07:00:30 UTC #10

I think hieuiph is referring to another post in which just the console output was given .

rknimmakayala 2017-03-23 07:01:24 UTC #11

Are you able to query your indexes in elastic search without creating a enricher ?

hieuiph 2017-03-23 07:39:20 UTC #12

Hi,

I tried without enricher:

and the result is same:

My ES configuration: I don't choose "Ingore Emitter".

hieuiph 2017-03-23 09:46:33 UTC #13

I tried view the ES output in View Data but I don't know how to choose "Index Type" filed.

any suggestion ?

thanks

anupam.jain 2017-03-23 10:34:51 UTC #14

Uncheck Ignore Emitter checkbox will actually save data to ES during inspect. The output for ES is blank, because it is the last component and there is no sense to show output of end ones.

The "Index Type" field is group name in lower case which is networkstats in your case