Spring Data Neo4j doesn't map Node List to record

Question

Objective

I am trying to get the longest shortest Paths between Service nodes in the Graph. There is some additional filters I used, but they don't change the result.

I also want to avoid handmapping the values since the ServiceEntity stores over 10 values, some of which are lists, and it may and probably will be expanded in the future.

Setup

I used the following Query,

WITH 
    $minLength AS minLength
    $projection AS projection


CALL gds.alpha.allShortestPaths.stream(projection)  
YIELD sourceNodeId, targetNodeId, distance  
WHERE gds.util.isFinite(distance) = true  

MATCH (source) WHERE source.id = sourceNodeId
MATCH (target) WHERE target.id = targetNodeId


// Filter by Service label, New Application and minimum Length  
WITH source, target, distance  
WHERE
  'Service' IN labels(source)
  AND 'Service' IN labels(target)
  AND source.`Service Type` = 'New Application'
  AND target.`Service Type` = 'New Application'
  AND source <> target
  AND distance >= minLength


// Get paths and transform to shortest paths  
MATCH path=shortestPath((source)-[*..30]->(target))  
WITH distance, [node in nodes(path) WHERE 'Service' IN labels(node)] AS servicesPath


// Return accordingly  
RETURN servicesPath  
ORDER BY distance DESC
LIMIT $limit

with the following call using the Neo4jClient.

public List<EntityPath> allPairsShortestPathsByLength(String projection, int minLength,
    int limit) {
  Map<String, Object> params =
      Map.of("projection", projection, "minLength", minLength, "limit", limit);
  Collection<EntityPath> all =
      client.query(APSPbyLengthQuery).bindAll(params).fetchAs(EntityPath.class).all();

  all.forEach(System.err::println);

  return List.copyOf(all);
}

The Neo4jClient is simply constructor injected.

I tried using a Neo4jTemplate instead as well, but I kept getting two different errors, which were

Caused by: org.springframework.core.convert.ConverterNotFoundException: No converter found capable of converting from type [org.neo4j.driver.internal.value.ListValue] to type [dpkass.tadafur.manjam.persistence.DTOs.paths.EntityPath]

or

Caused by: org.neo4j.driver.exceptions.value.NotMultiValued: NODE is not a keyed collection

The EntityPath simply takes a List<ServiceEntity> as it's parameter.

public record EntityPath(List<ServiceEntity> servicesPath)

To keep this short the ServiceEntity is relatively complex. It stores a lot of values, with connections to other Nodes. But since I've been using the same entity to import it should be alright.

Note: Every class that has something with Entity in it's name is a persistence DTO.

Approaches

Neo4jRepository

I couldn't use the builtin Neo4jRepository, because of two reasons:

1. The EntityPath I am using doesn't have an ID (because it's not a domain object). Hence Spring didn't let me use it as the type parameter i. e. Neo4jRepository<EntityPath, Long>
2. If I simply used the ServiceEntity as the type (which I have used in my import service), it tries to map the reuslts to a List (which I found out by debugging.

This would be the best approach of course since it minimizes my medling in the conversion. This was my Repositroy:

@Repository
public interface PathsSpringRepository extends Neo4jRepository<Object, Long> {

  @Query("""
      WITH $minLength as minLength
            
      CALL gds.alpha.allShortestPaths.stream($projection)
      YIELD sourceNodeId, targetNodeId, distance
      WHERE gds.util.isFinite(distance) = true
            
      MATCH (source) WHERE id(source) = sourceNodeId
      MATCH (target) WHERE id(target) = targetNodeId
            
            
      // Filter by Service label, New Application and min Dist
      WITH source, target, distance
      WHERE
        'Service' IN labels(source)
        AND 'Service' IN labels(target)
        AND source.`Service Type` = 'New Application'
        AND target.`Service Type` = 'New Application'
        AND source <> target
        AND distance >= minLength
            
            
      // Get paths and transform to shortest paths
      MATCH path=shortestPath((source)-[*]-(target))
      WITH source, target, distance, [node in nodes(path) WHERE 'Service' IN labels(node)] AS servicesPath
            
            
      // Return accordingly
      RETURN source, target, distance, servicesPath
      ORDER BY distance DESC
      LIMIT $limit
      """)
  List<EntityPathDescription> allPairsShortestPathsByLength(String projection, int minLength,
      int limit);

  @Query("""
      CALL gds.graph.project(
        $name,
        ['Service', 'Artifact'],
        ['is needed for', 'gives']
      )
      YIELD graphName
      """)
  String createProjection(String name);
}

Neo4jTemplate

As I've already said, I've tried using the template instead, but it has the same issues. I don't have the code anymore, but I basically just called findAll with EntitiyPath.class as domainType.

Using a mapper ListValue to EntityPath

Here I had two different tries. Again of course I don't want to hand map everything. So I consulted ChatGPT and he said I can use the internal ConverterService bean. So I tried this:

@Component
public class ListValueToEntityPathConverter implements Converter<ListValue, EntityPath> {

  private final ConversionService conversionService;

  public ListValueToEntityPathConverter(ConversionService conversionService) {
    this.conversionService = conversionService;
  }

  @Override
  public EntityPath convert(ListValue source) {
    var stream = StreamSupport.stream(source.values().spliterator(), false);
    List<ServiceEntity> serviceEntities =
        stream.map(value -> conversionService.convert(value.asObject(), ServiceEntity.class))
              .collect(Collectors.toList());
    return new EntityPath(serviceEntities);
  }
}

When I tried this I still got the converter missing error I meantioned above. ChatGPT came up with the idea of manually configuring it. My second try was the same converter, using @Lazy on the injection and configuring it using the converter factory. And still the same error, saying the converter is missing.

Final note

As I am writing this down I am realizing that the missing nodes i. e. the nodes connected to the ServiceEntity, that I didn't get back from the query, could be the issue. Tomorrow I will try writing a custom projection so that I only keep the values that I need.

I will still be leaving this up if anyone else has any suggestions how to resolve the issue.

Edit

I added the projection called SimplifiedService and reverted back to a Neo4jRepository.

public record SimplifiedServiceEntity(@Id Long id, @Property("name") String name)

It still won't map to EntityPath. So I tried removing that abstraction (for now) and simply mapping to a List<SimplifiedService>. Consequently, the PathsSpringRepository returns a List<List<SimplifiedServiceEntity>>.

Issues

Depending on the @Query I use I get either a Mapping Exception or a false result.

I merely changed the return and transform part of the query:

// Get paths and transform to shortest paths
// Get paths and transform to shortest paths
MATCH path=shortestPath((source)-[*..30]->(target))
WITH
  distance,
  [node in nodes(path) WHERE 'Service' IN labels(node)] AS servicesPath
ORDER BY distance DESC
LIMIT $limit
      
// Return accordingly
RETURN collect(servicesPath)

or the same, but omitting collect, i. e.

RETURN servicesPath

Pretermitting collect

When I do this and collect into a list of lists, I don't get an exception. However, what I do get is a list of 2807 lists, each is a single node. If I put them into a Set, I get a set of size 318.

I went on a hunt, to find out what these numbers meant. Basically, if you would concatenate all the rows you get from the query you will get a list of 2807 nodes and if you use DISTINCT 318 remain.

Somehow, SDN fails to recognize the rows as lists, making the whole structure a list of lists of nodes, which is exactly what we need.

Using collect

When I use collect I get the following Exception:

Caused by: org.springframework.data.mapping.MappingException: More than one matching node in the record

But I realize that this is the way it is supposed to work (I guess). So I am building on that.

It probably is not sure if it should map it as in the no-collect version or as it is supposed to be. I don't know how to make it choose between one of them, or rather the right one, (again) without using my own Mapper.