After upgrading plugins from Kong 2.2.1 to Kong 3.3, under the following conditions:
- I have a sample piece of lua code in a custom plugin that utilizes the kong cache function
`local credential_cache_key = kong.db.oauth2_credentials:cache_key(client_id)
client, err = kong.cache:get(credential_cache_key, nil, load_oauth2_credential_by_client_id, client_id)`
where the callback function to retrieve the entity to be loaded into the cache is a dao select function
local function load_oauth2_credential_by_client_id(client_id)
local credential, err = kong.db.oauth2_credentials:select_by_client_id(client_id)
if err then
return nil, err
end
return credential
end
- Kong is deployed with the custom plugin in Kubernetes with more than one instance.
More often than not, after the initial start up and during the plugin execution, kong will hit the following error
[error] 1261#0: *4864420 [kong] init.lua:359 [partner-custom-oauth2] /opt/kong/plugins/partner-custom-oauth2/access.lua:1071: attempt to index local 'client' (a nil value)
indicating that the value in the cache/entity returned from the select statement is nil, even though a manual db query indicates that the entity exists but ws_id is null.

After a few kong restarts, it is noticed that the error goes away and when checking the oauth2_credentials table the ws_id is populated.
When turning on OpenTelemetry logs, it is noted that the dao select statement queries ws_id as well, and having ws_id as null could then be an issue
SELECT id, EXTRACT(EPOCH FROM created_at AT TIME ZONE 'UTC') AS created_at, expires_in, access_token, ws_id, FLOOR(EXTRACT(EPOCH FROM (ttl AT TIME ZONE 'UTC' - CURRENT_TIMESTAMP AT TIME ZONE 'UTC'))) AS ttl .... (columns scrubbed to remove sensitive information) FROM oauth2_tokens WHERE access_token = 'jkCFYnyzN2gumQI9lbKgYTuHE63XNXDO' AND (ttl IS NULL OR ttl >= CURRENT_TIMESTAMP AT TIME ZONE 'UTC') AND (ws_id = '6c431e2e-020c-4e4c-af58-a25f21b2d5de') LIMIT 1;
Anyone has had a similar issue or know the cause? What causes the ws_id to be null in the first place..