Elasticsearch doesn't allow me to add long dict [TransportError(400)]

Question

I am using elasticsearch-py for adding some dict to the index in elastic, I added dictionary to index and dictionary with lists in it, in my code was adding this dictionary but it give me the error like this: RequestError: TransportError(400, u'mapper_parsing_exception', u'failed to parse [long_spec]') after search I got that I should add some maps in elastic and then I add "long_spec":{ "type": "text" } and in other test "long_spec"{ "type": "string" } in mapping but it doesn't work.

I use this in my python self.es.index(index="new_index", doc_type='user',body=dict) and my dict is like this:

{'long_spec': [{'header': u'Design', 'specification': [{u'Device type:': [u'Smart phone']}, {u'OS:': [u'Android (7.0)']}, {u'Form factor:': [u'Candybar']}, {u'Dimensions:': [u'6.06 x 3.03 x 0.35 inches (153.9 x 77 x 8.9 mm)']}, {u'Weight:': [u'6.38 oz (181 g)']}, {u'Features:': [u'Fingerprint (touch), Touch sensitive control keys']}, {u'Keys:': [u'Right: Volume control, Lock/Unlock key']}, {u'Colors:': [u'Black, Gold, Pink']}]}, {'header': u'Display', 'specification': [{u'Physical size:': [u'5.5 inches']}, {u'Resolution:': [u'720 x 1280 pixels']}, {u'\n': [u'267 ppi']}, {u'\n': [u'IPS LCD']}, {u'\n': [u'70.40 %']}, {u'Colors:': [u'16 777 216']}, {u'Touchscreen:': [u'Capacitive, Multi-touch', u'\n']}, {u'Features:': [u'Ambient light sensor, Proximity sensor']}]}, {'header': u'Camera', 'specification': [{u'Camera:': [u'13 megapixels', u'\n', u'LED', u'\n', u'F2.0', u'\n', u'25 mm', u'\n', u'Dual-camera (13-megapixel, 120-degree viewing angle), Autofocus (Phase detection)', u'\n', u'Touch to focus, Face detection, Self-timer, Digital zoom, Geo tagging', u'\n', u'Exposure compensation, ISO control, White balance presets', u'\n', u'Burst mode, High Dynamic Range mode (HDR), Panorama', u'\n']}, {u'Camcorder:': [u'1920x1080 (1080p HD) (30 fps)', u'\n', u'Time-lapse video, Picture-taking during video recording, Digital image stabilization, Video calling, Video sharing', u'\n']}, {u'Front-facing camera:': [u'8 megapixels', u'\n', u'Wide Angle, Flash', u'\n']}]}, {'header': u'Hardware', 'specification': [{u'\n': [u'Qualcomm Snapdragon 430 8937 ']}, {u'\n': [u'Octa-core, 1400 MHz, ARM Cortex-A53, 64-bit, 28 nm']}, {u'\n': [u'Adreno 505', u'\n']}, {u'\n': [u'4 GB RAM (800 MHz) / LPDDR2']}, {u'\n': [u'64 GB']}, {u'\n': [u'microSD, microSDHC, microSDXC up to 256 GB']}]}, {'header': u'Battery', 'specification': [{u'\n': [u'5000 mAh']}, {u'Type:': [u'Not user replaceable']}, {u'Fast Charge:': [u'Yes']}, {u'Talk time (3G):': [u'40.00 hours']}, {u'Stand-by time (4G):': [u'46.0 days (1104 hours)']}, {u'Internet use:': [u'26 hours', u'\n']}, {u'Video playback:': [u'22.00 hours']}]}, {'header': u'Multimedia', 'specification': [{u'Screen mirroring:': [u'Wireless screen share']}, {u'Music player:': [u'Album, Artist, Genre, Playlists', u'\n', u'Album art cover, Background playback', u'\n', u'MP3, AAC, FLAC, WAV, M4A (Apple lossless), AMR, OGG, MIDI', u'\n']}, {u'Video playback:': [u'MPEG4, H.263, H.264', u'\n']}, {u'Radio:': [u'FM', u'\n']}, {u'Speakers:': [u'Earpiece, Loudspeaker']}, {u'Headphones connector:': [u'3.5mm']}]}, {'header': u'Cellular', 'specification': [{u'GSM:': [u'850, 900, 1800, 1900 MHz']}, {u'UMTS:': [u'800, 850, 900, 1700/2100, 1900, 2100 MHz']}, {u'LTE (FDD):': [u'Bands 1, 2, 3, 4, 5, 7, 8, 12, 17, 18, 19, 26, 28']}, {u'LTE (TDD):': [u'Bands 38, 41']}, {u'Data:': [u'LTE Cat 4 (150/50 Mbit/s), HSDPA+ (4G) 42.2 Mbit/s, HSUPA 5.76 Mbit/s']}, {u'2G Data:': [u'EDGE, GPRS']}, {u'Nano SIM:': [u'Yes']}, {u'Multiple SIM cards:': [u'2 slots']}, {u'VoLTE:': [u'Yes']}, {u'Global Roaming:': [u'Yes']}]}, {'header': u'Phone features', 'specification': [{u'Sensors:': [u'Accelerometer, Gyroscope, Compass', u'\n']}, {u'Notifications:': [u'Haptic feedback']}, {u'Phonebook:': [u'Unlimited entries, Caller groups, Multiple numbers per contact, Search by both first and last name, Picture ID, Ring ID']}, {u'Organizer:': [u'Calendar, Alarm, Calculator, World clock, Timer, Stopwatch, Countdown timer']}, {u'Messaging:': [u'SMS, MMS, Threaded view, Predictive text input']}, {u'E-mail:': [u'IMAP, POP3, SMTP, Microsoft Exchange', u'\n']}, {u'Instant Messaging:': [u'Hangouts / Google Talk', u'\n']}, {u'Other features:': [u'Voice dialing, Voice commands, Voice recording']}]}, {'header': u'Connectivity', 'specification': [{u'Bluetooth:': [u'4.1', u'\n', u'Advanced Audio Distribution (A2DP), Audio/Visual Remote Control Profile (AVRCP), Generic Access (GAP), Handsfree (HFP), Headset (HSP), Message Access Profile (MAP), Object Push (OPP), Phone Book Access (PBAP)', u'\n']}, {u'Wi-Fi:': [u'802.11 b, g, n', u'\n', u'Yes', u'\n', u'Yes', u'\n']}, {u'USB:': [u'USB 2.0', u'\n', u'microUSB', u'\n', u'Mass storage device, USB Host, USB charging', u'\n']}, {u'Charging connector:': [u'microUSB']}, {u'\n': [u'GPS, A-GPS, Glonass, BeiDou']}, {u'Navigation:': [u'Turn-by-turn navigation, Voice navigation']}, {u'Other:': [u'VoIP, Tethering, Computer sync, OTA sync']}]}, {'header': u'Availability', 'specification': [{u'Officially announced:': [u'17 Aug 2017']}]}]}

Answer 1

You'll get mapper_parsing_exception whenever the field you're trying to index is of different type than the mappings specified. In your case, the long_spec dictionary must have different data type across various documents and hence the error. You can replicate the error by following the below:

Try to index a document

PUT regexpindex/product/4
{ 
    "documentId": 1234,
    "someOtherInfo": "Product1\\Product1",
    "permissions": ["a", "b", "c", "g"]
}

Try to index another document with a string data type for documentId field

PUT regexpindex/product/4
{ 
    "documentId": "some-string",
    "someOtherInfo": "Product1\\Product1",
    "permissions": ["a", "b", "c", "g"]
}

and you'll get mapper_parsing_exception.

Please make sure that all your documents have the same data type across all the fields. You can also ignore the documents having different datatype if you think those documents are malformed. Read more about it here.