Django bulk_create() with inexistent ForeignKey

62 views Asked by At

I have 2 models. For simplicity, I deleted all unnecessary fields.

class User(models.Model):
    id = models.CharField(max_length=36, primary_key=True, null=False)
    age = models.IntegerFiels(default=-1)

and

class Device(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE)

Through the API, I receive information about devices and create a list with devices.

devices = []
for device_data in data:
    devices.append(
        Device(
            user_id=device_data("user_id")
        )
    )

Once I have the list of devices, I add it to the database.

Device.objects.bulk_create(devices, ignore_conflicts=True)

The problem is that if, when creating a Device, I specified a user_id that is not in the database, I will get a crash because I am trying to link with a non-existent id.

Now to get around this I do the following:

devices = []
users = []
for device_data in data:
    users.append(
        User(
            id=device_data("user_id", age = -1)
        )
    )
    devices.append(
        Device(
            user_id=device_data("user_id")
        )
    )

Then, when saving:

User.objects.bulk_create(users, ignore_conflicts=True)
Device.objects.bulk_create(devices, ignore_conflicts=True)
User.objects.filter(age=-1).delete()

How can I make sure that when saving, if the specified user_id does not exist, this device is not added?

UPDATED:

Another solution is to change the field

User.age = models.IntegerFiels(null=True) 

and

 Device.user = models.ForeignKey(User, on_delete=models.CASCADE, null=True)

After this, I can save devices without users.

Device.objects.bulk_create(devices, ignore_conflicts=True)

After which, do: Device.objects.filter(user__age__isnull=True).delete()

This works, but I don’t like the fact that you need to check for age, and not for the user itself.

2

There are 2 answers

5
CoffeeBasedLifeform On BEST ANSWER

The problem is that if, when creating a Device, I specified a user_id that is not in the database, I will get a crash because I am trying to link with a non-existent id.

If you don't want your devices to require a User, then make the ForeignKey nullable:

class Device(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE, null=True, blank=true)

UPDATE
Oh I misunderstood. You want to bulk create only those Devices that have an existing user id? Why not just filter out the device data with usable user_id?

devices = []
user_ids = User.objects.values_list("id", flat=True)
# assuming that device_data is a dictionary
for device_data in data:
    if "user_id" in device_data and device_data["user_id"] in user_ids:
        devices.append(Device(**device_data)) 
4
Hashem On

I have a few suggestions for you

1 - apply a query to check if user is exists or not

devices = []
for device_data in data:
user_id = device_data.get("user_id")  # Use get() to avoid KeyError if user_id is not present
if User.objects.filter(id=user_id).exists():
    device = Device(user_id=user_id)
    devices.append(device)

and that will be expensive operation if you have a big data

2 - apply Try Catch block with non bulk operation with simple for loop, and you can use a background task to improve it if that meet system requirements

3 - apply filter when you are getting data from API if you can to get only common values

4 - make user in Device model accept Null value and make it as a default value if there is no data and in this way you can keep all data if you need it in feature to analysis it and you can control how you will get it using filter