redisgraph-bulk-loader returns with IndexError

Question

I am trying to use the redisgraph-bulk-loader to load the dataset into the RedisGraph instance running in a docker container. However, it returns with IndexError: list index out of range which I do not understand. I have read the documentation and my CSV files are valid.

I have opened an issue here.

Command : $ redisgraph-bulk-loader DemoGraph --enforce-schema --nodes-with-label TBox import/nodes.csv --relations-with-type relations import/relationships.csv

Log :

Traceback (most recent call last):
  File "/home/raj/Desktop/2020-msc-raj-dharmendra-joshi/src/.env/bin/redisgraph-bulk-loader", line 33, in <module>
    sys.exit(load_entry_point('redisgraph-bulk-loader==0.9.6', 'console_scripts', 'redisgraph-bulk-loader')())
  File "/home/raj/Desktop/2020-msc-raj-dharmendra-joshi/src/.env/lib/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/raj/Desktop/2020-msc-raj-dharmendra-joshi/src/.env/lib/python3.9/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/raj/Desktop/2020-msc-raj-dharmendra-joshi/src/.env/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/raj/Desktop/2020-msc-raj-dharmendra-joshi/src/.env/lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/raj/Desktop/2020-msc-raj-dharmendra-joshi/src/.env/lib/python3.9/site-packages/redisgraph_bulk_loader/bulk_insert.py", line 117, in bulk_insert
    reltypes = parse_schemas(RelationType, query_buf, relations, relations_with_type, config)
  File "/home/raj/Desktop/2020-msc-raj-dharmendra-joshi/src/.env/lib/python3.9/site-packages/redisgraph_bulk_loader/bulk_insert.py", line 23, in parse_schemas
    schemas[idx + offset] = cls(query_buf, csv_tuple[1], csv_tuple[0], config)
  File "/home/raj/Desktop/2020-msc-raj-dharmendra-joshi/src/.env/lib/python3.9/site-packages/redisgraph_bulk_loader/relation_type.py", line 11, in __init__
    super(RelationType, self).__init__(infile, type_str, config)
  File "/home/raj/Desktop/2020-msc-raj-dharmendra-joshi/src/.env/lib/python3.9/site-packages/redisgraph_bulk_loader/entity_file.py", line 189, in __init__
    self.convert_header() # Extract data from header row.
  File "/home/raj/Desktop/2020-msc-raj-dharmendra-joshi/src/.env/lib/python3.9/site-packages/redisgraph_bulk_loader/entity_file.py", line 260, in convert_header
    self.convert_header_with_schema(header)
  File "/home/raj/Desktop/2020-msc-raj-dharmendra-joshi/src/.env/lib/python3.9/site-packages/redisgraph_bulk_loader/entity_file.py", line 238, in convert_header_with_schema
    col_type = convert_schema_type(pair[1].upper().strip())
IndexError: list index out of range

nodes.csv :

"id:ID","label:STRING","properties__unique:STRING","properties__namespace:STRING","properties__description:STRING","properties__title:STRING","properties__uri:STRING","properties__xsd_type:STRING","properties__identifier:STRING","properties__ontology_level:STRING","properties__html_info:STRING","properties__admin:string","properties__sing:string","properties__pl:string","properties__version:string","properties__comment:string","properties__url:string"
"0","TBox","False","dcterms","Date of creation of the resource.","created","http://example.org/tbox/created","xsd:dateTimeStamp","","","","","","","","",""
"1","TBox","False","dcterms","A URI that identifies a party or group who can access the resource or an indication of its security status.","accessrights","http://example.org/tbox/accessrights","xsd:anyURI","","","","","","","","",""
"2","Abox","","","A Sensor Collar","SensorCollar1","","","http://example.org/abox/SensorCollar8153e23d","","","","","","","",""
"3","Abox","","","A Herd","Herd1","","","http://example.org/abox/Herdae1a69e9","","","","","","","",""
"4","Abox","","","A Cow","Cow1","","","http://example.org/abox/Cowebcc054f","","","","","","","",""
"5","Abox","","","A IMU Sensor","IMUSensor1","","","http://example.org/abox/IMUSensorf9404e5d","","","","","","","",""
"6","Abox","","","An activity","Sensing2","","","http://example.org/abox/Sensingae00225d","","","","","","","",""
"7","Abox","","","An activity","Sensing1","","","http://example.org/abox/Sensingc772a8a1","","","","","","","",""
"8","Abox","","","A query","Query1","","","http://example.org/abox/Query39192292","","","","","","","",""
"9","Abox","","","A query","Query2","","","http://example.org/abox/Querybc3837d3","","","","","","","",""
"10","Abox","","","A query","Query3","","","http://example.org/abox/Queryad1015ac","","","","","","","",""
"11","Abox","","","A query","Query4","","","http://example.org/abox/Queryf1f13065","","","","","","","",""
"12","Abox","","","An IMUSensro data","IMUData1","","","http://example.org/abox/IMUSensorData022035ed","","","","","","","",""
"13","Abox","","","Video data generated from vedio sensing","VideoRecording1","","","http://example.org/abox/VideoDatadb84cb6d","","","","","","","",""
"14","Abox","","","An activity which represents an expert stays at farm to collect data","FarmStay1","","","http://example.org/abox/FarmStayb34f5094","","","","","","","",""
"15","Abox","","","An agent or person","Lara","","","http://example.org/abox/Person7bcedef0","","","","","","","",""
"16","Abox","","","An activity that happens in the farm for collecting groundtruth data","GroundTruthGeneration1","","","http://example.org/abox/GroundTruthGenerationf6f3135b","","","","","","","",""
"17","Abox","","","a DataSet that contains the ground truth generated from video files","GroundTruth1","","","http://example.org/abox/GroundTruth949a4f3a","","","","","","","",""
"18","Abox","","","A specified set of data that is stored in a data store and can be retrieved using a query.","Database1","","","http://example.org/abox/DataSet743f627d","","","","","","","",""
"19","Abox","","","An activity for converting raw data into","DataProcessing1","","","http://example.org/abox/DataProcessing7c27853b","","","","","","","",""
"20","Abox","","","A system that represents an algorithm that can manipulate data ","

relationship.csv :

":START_ID","label:STRING","start__labels:STRING",":END_ID","end__labels__001:STRING","properties__level:STRING","properties__namespace:STRING","properties__description:STRING","properties__title","properties__uri","end__labels__002"
"2","type_of","Abox","110","TBox","","","","","",""
"3","type_of","Abox","89","TBox","","","","","",""
"4","type_of","Abox","88","TBox","","","","","",""
"3","hasPart","Abox","4","Abox","","","","","",""
"4","relation","Abox","2","Abox","","","","","",""
"5","type_of","Abox","108","TBox","","","","","",""
"2","hasPart","Abox","5","Abox","","","","","",""
"6","type_of","Abox","82","TBox","","","","","",""
"7","type_of","Abox","82","TBox","","","","","",""
"8","type_of","Abox","73","TBox","","","","","",""
"9","type_of","Abox","73","TBox","","","","","",""
"10","type_of","Abox","73","TBox","","","","","",""
"11","type_of","Abox","73","TBox","","","","","",""
"12","type_of","Abox","103","TBox","","","","","",""
"12","requires","Abox","11","Abox","","","","","",""
"12","wasGeneratedBy","Abox","6","Abox","","","","","",""
"6","wasAssociatedWith","Abox","5","Abox","","","","","",""
"13","type_of","Abox","102","TBox","","","","","",""
"14","type_of","Abox","81","TBox","","","","","",""
"15","type_of","Abox","76","TBox","","","","","",""
"14","wasAssociatedWith","Abox","15","Abox","","","","","",""
"7","wasAssociatedWith","Abox","15","Abox","","","","","",""
"13","requires","Abox","10","Abox","","","","","",""
"13","wasGeneratedBy","Abox","7","Abox","","","","","",""
"16","type_of","Abox","80","TBox","","","","","",""
"16","used","Abox","13","Abox","","","","","",""
"16","wasAssociatedWith","Abox","15","Abox","","","","","",""
"14","hasPart","Abox","7","Abox","","","","","",""
"7","wasAssociatedWith","Abox","3","Abox","","","","","",""
"17","type_of","Abox","95","TBox","","","","","",""
"18","type_of","Abox","72","TBox","","","","","",""
"25","subclass_of","TBox","24","TBox","","","","","",""
"26","subclass_of","TBox","24","TBox","","","","","",""
"27","subclass_of","TBox","24","TBox","","","","","",""
"70","subclass_of","TBox","27","TBox","","","","","",""
"71","subclass_of","TBox","70","TBox","","","","","",""
"72","subclass_of","TBox","70","TBox","","","","","",""
"73","subclass_of","TBox","70","TBox","","","","","",""
"19","type_of","Abox","79","TBox","","","","","",""
"20","type_of","Abox","111","TBox","","","","","",""
"19","wasAssociatedWith","Abox","20","Abox","","","","","",""
"19","used","Abox","17","Abox","","","","","",""
"18","wasGeneratedBy","Abox","19","Abox","","","","","",""
"18","requires","Abox","9","Abox","","","","","",""
"19","used","Abox","12","Abox","","","","","",""
"17","wasGeneratedBy","Abox","16","Abox","","","","","",""
"17","requires","Abox","8","Abox","","","","","",""
"21","type_of","Abox","85","TBox","","","","","",""
"22","type_of","Abox","86","TBox","","","","","",""
"23","type_of","Abox","101","TBox","","","","","",""
"28","type_of","Abox","83","TBox","","","","","",""
"29","type_of","Abox","95","TBox","","","","","",""
"30","type_of","Abox","77","TBox","","","","","",""
"21","used","Abox","12","Abox","","","","","",""
"31","type_of","Abox","93","TBox","","","","","",""
"21","wasAssociatedWith","Abox","31","Abox","","","","","",""
"22","wasAssociatedWith","Abox","31","Abox","","","","","",""
"22","used","Abox","23","Abox","","","","","",""
"23","wasGeneratedBy","Abox","28","Abox","","","","","",""
"28","used","Abox","29","Abox","","","","","",""
"28","wasassociatedWith","Abox","30","Abox","","","","","",""
"29","requires","Abox","9","Abox","","","","","",""
"17","requires","Abox","8","Abox","","","","","",""
"21","used","Abox","17","Abox","","","","","",""
"32","type_of","Abox","98","TBox","","","","","",""
"33","type_of","Abox","98","TBox","","","","","",""
"74","subclass_of","TBox","25","TBox","","","","","",""
"75","subclass_of","TBox","25","TBox","","","","","",""
"76","subclass_of","TBox","25","TBox","","","","","",""
"77","subclass_of","TBox","76","TBox","","","","","",""
"78","subclass_of","TBox","26","TBox","","","","","",""
"79","subclass_of","TBox","26","TBox","","","","","",""
"80","subclass_of","TBox","26","TBox","","","","","",""
"81","subclass_of","TBox","26","TBox","","","","","",""
"82","subclass_of","TBox","26","TBox","","","","","",""
"83","subclass_of","TBox","26","TBox","","","","","",""
"84","subclass_of","TBox","26","TBox","","","","","",""
"85","subclass_of","TBox","26","TBox","","","","","",""
"86","subclass_of","TBox","26","TBox","","","","","",""
"87","subclass_of","TBox","26","TBox","","","","","",""
"88","subclass_of","TBox","25","TBox","","","","","",""
"89","subclass_of","TBox","25","TBox","","","","","",""
"90","subclass_of","TBox","70","TBox","","","","","",""
"91","subclass_of","TBox","111","TBox","","","","","",""
"93","subclass_of","TBox","111","TBox","","","","","",""
"94","subclass_of","TBox","111","TBox","","","","","",""
"95","subclass_of","TBox","72","TBox","","","","","",""
"96","subclass_of","TBox","72","TBox","","","","","",""
"97","subclass_of","TBox","72","TBox","","","","","",""
"98","subclass_of","TBox","72","TBox","","","","","",""
"99","subclass_of","TBox","72","TBox","","","","","",""
"100","subclass_of","TBox","72","TBox","","","","","",""
"101","subclass_of","TBox","71","TBox","","","","","",""
"102","subclass_of","TBox","71","TBox","","","","","",""
"103","subclass_of","TBox","72","TBox","","","","","",""
"104","subclass_of","TBox","27","TBox","","","","","",""
"33","wasGeneratedBy","Abox","21","Abox","","","","","",""
"28","used","Abox","33","Abox","","","","","",""
"34","type_of","Abox","84","TBox","","","","","",""
"34","wasAssociatedWith","Abox","31","Abox","","","","","",""
"35","type_of","Abox","97","TBox","","","","","",""
"35","wasGeneratedBy","Abox","22","Abox","","","","","",""
"34","used","Abox","35","Abox","","","","","",""
"32","wasGeneratedBy","Abox","34","Abox","","","","","",""
"36","type_of","Abox","103","TBox","","","","","",""
"34","used","Abox","36","Abox","","","","","",""
"105","subclass_of","TBox","93","TBox","","","","","",""
"107","subclass_of","TBox","114","TBox","","","","","",""
"108","subclass_of","TBox","114","TBox","","","","","",""
"109","subclass_of","TBox","114","TBox","","","","","",""
"110","subclass_of","TBox","114","TBox","","","","","",""
"111","subclass_of","TBox","75","TBox","","","","","",""
"112","subclass_of","TBox","75","TBox","","","","","",""
"113","subclass_of","TBox","93","TBox","","","","","",""
"114","subclass_of","TBox","75","TBox","","","","","",""
"24","object_property","TBox","24","TBox","upper","dcterms","A related KBMSThing  that is referenced, cited, or otherwise pointed to by the described SKMThing","references","http://example.org/tbox/references",""
"24","object_property","TBox","24","TBox","upper","dcterms","A relation between two KBMSThings (bi-directional version of references)","relation","http://example.org/tbox/relation",""
"24","object_property","TBox","24","TBox","upper","dcterms","A related KBMSThing that is included either physically or logically in the described SKMThing.","haspart","http://example.org/tbox/haspart",""
"24","object_property","TBox","24","TBox","upper","dcterms","A related resource that is supplanted, displaced, or superseded by the described resource.","replaces","http://example.org/tbox/replaces",""
"70","object_property","TBox","71","TBox","upper","dcterms","The Document includes  a statement about various property rights associated with the SKMThing, including intellectual property rights.","rights","http://example.org/tbox/rights",""
"70","object_property","TBox","25","TBox","upper","dcterms","A person or organization owning or managing rights over the SKMThing.","rightsholder","http://example.org/tbox/rightsholder",""
"24","object_property","TBox","25","TBox","upper","dcterms","A user that created the instance in the SKM system.","creator","http://example.org/tbox/creator",""
"24","object_property","TBox","71","TBox","upper","dcterms","The related Document describes subunits of the resource.","tableofcontents","http://example.org/tbox/tableofcontents",""
"26","object_property","TBox","76","TBox","upper","foaf","This Person is a member of this Project.","member","http://example.org/tbox/member",""
"74","object_property","TBox","76","TBox","upper","foaf","This Person is a member of this Organization.","member","http://example.org/tbox/member",""
"70","object_property","TBox","73","TBox","upper","dcterms","A related resource that is required by the described data to support its function, delivery, or coherence.","requires","http://example.org/tbox/requires",""
"27","object_property","TBox","26","TBox","upper","prov","Generation is the completion of production of a new entity by an activity. This entity did not exist before generation and becomes available for usage after this generation.","wasgeneratedby","http://example.org/tbox/wasgeneratedby",""
"27","object_property","TBox","27","TBox","upper","prov","A derivation is a transformation of an entity into another, an update of an entity resulting in a new one, or the construction of a new entity based on a pre-existing entity.","wasderivedfrom","http://example.org/tbox/wasderivedfrom",""
"27","object_property","TBox","25","TBox","upper","prov","Attribution is the ascribing of an entity to an agent.","wasattributedto","http://example.org/tbox/wasattributedto",""
"26","object_property","TBox","27","TBox","upper","prov","Usage is the beginning of utilizing an entity by an activity. Before usage, the activity had not begun to utilize this entity and could not have been affected by the entity.","used","http://example.org/tbox/used",""
"26","object_property","TBox","26","TBox","upper","prov","Communication is the exchange of an entity by two activities, one activity using the entity generated by the other.","wasinformedby","http://example.org/tbox/wasinformedby",""
"26","object_property","TBox","25","TBox","upper","prov","An activity association is an assignment of responsibility to an agent for an activity, indicating that the agent had a role in the activity. It further allows for a plan to be specified, which is the plan intended by the agent to achieve some goals in the context of this activity.","wasassociatedwith","http://example.org/tbox/wasassociatedwith",""
"25","object_property","TBox","25","TBox","upper","prov","Delegation is the assignment of authority and responsibility to an agent (by itself or by another agent) to carry out a specific activity as a delegate or representative, while the agent it acts on behalf of retains some responsibility for the outcome of the delegated work. For example, a student acted on behalf of his supervisor, who acted on behalf of the department chair, who acted on behalf of the university; all those agents are responsible in some way for the activity that took place but we do not say explicitly who bears responsibility and to what degree.","actedonbehalfof","http://example.org/tbox/actedonbehalfof",""
"24","required_property","TBox","126","TBox","","","","","","property"
"24","required_property","TBox","127","TBox","","","","","","property"
"24","required_property","TBox","125","TBox","","","","","","property"
"77","required_property","TBox","135","TBox","","","","","","property"
"24","optional_property","TBox","129","TBox","","","","","","property"
"24","optional_property","TBox","138","TBox","","","","","","property"
"24","optional_property","TBox","123","TBox","","","","","","property"
"26","optional_property","TBox","137","TBox","","","","","","property"
"26","optional_property","TBox","136","TBox","","","","","","property"
"70","optional_property","TBox","124","TBox","","","","","","property"
"70","optional_property","TBox","140","TBox","","","","","","property"
"70","optional_property","TBox","123","TBox","","","","","","property"
"73","optional_property","TBox","132","TBox","","","","","","property"
"73","optional_property","TBox","131","TBox","","","","","","property"
"73","optional_property","TBox","139","TBox","","","","","","property"
"76","optional_property","TBox","133","TBox","","","","","","property"
"76","optional_property","TBox","134","TBox","","","","","","property"

Answer 1

The error you're encountering occurs at this line - https://github.com/RedisGraph/redisgraph-bulk-loader/blob/master/redisgraph_bulk_loader/entity_file.py#L233 .

At this point, we are trying to retrieve the type specified after the colon in a specific field of a header line. As such, an IndexError indicates that no colon was found in one of the fields.

Looking at relationship.csv, I see that the header row ends with "properties__title","properties__uri","end__labels__002". These columns should have a colon-separated type, like the preceding fields.