Schemathesis issues with stateful testing using OpenAPI links

48 views Asked by At

We are using schemathesis for testing our API’s, we want to utilize it for stateful testing, hence we defined the links in our OpenAPI as follows:

Endpoint which will use link:

{"post":
     {
       "operationId":
       "createProfile","description":"",
       "parameters":[],
      "requestBody":{
         "content":{
            "application/json":{
               "schema":{
                  "$ref":"#/components/schemas/Profile"
               }
            },
            "application/x-www-form-urlencoded":{
               "schema":{
                  "$ref":"#/components/schemas/Profile "
               }
            },
            "multipart/form-data":{
               "schema":{
                  "$ref":"#/components/schemas/Profile"
               }
            }
         }
      },
      "responses":{
         "201":{
            "content":{
               "application/json":{
                  "schema":{
                     "$ref":"#/components/schemas/Profile"
                  }
               }
            },
            "links":{
               "GetProfileById":{
                  "operationId":"retrieveProfile",
                  "parameters":{
                     "id":"$response.body#/id"
                  },
                  "description":"The`id`value returned in the response can be used as the`id`parameter in`GET /profile/{id}`.\\n"
               }
            },
            "description":""
         }
      },
      "tags":[
         "profile"
      ]

}
}

The linked endpoint:

{
   "/profile/{id}":{
      "get":{
         "operationId":"retrieveProfile",
         "description":"",
         "parameters":[
            {
               "name":"id",
               "in":"path",
               "required":true,
               "description":"A unique integer value identifying this Profile.",
               "schema":{
                  "type":"string"
               }
            }
         ],
         "responses":{
            "200":{
               "content":{
                  "application/json":{
                     "schema":{
                        "$ref":"#/components/schemas/Profile"
                     }
                  }
               },
               "description":""
            }
         },
         "tags":[
            "profile"
         ]
      },
      "components":{
         "schemas":{
            "Profile":{
               "type":"object",
               "properties":{
                  "id":{
                     "type":"integer",
                     "readOnly":true
                  },
                  "name":{
                     "type":"string",
                     "maxLength":100
                  },
                  "afkorting":{
                     "type":"string",
                     "maxLength":10
                  }
               }
            }
         }
      }
   }
}

I have followed the example from:

https://schemathesis.readthedocs.io/en/stable/stateful.html#how-to-specify-connections and https://swagger.io/docs/specification/links/

I call it in following way:

headers = {
'Content-Type': 'application/json',
'Authorization': f'Basic {base64_encoded_auth_header}'
}

schema = schemathesis.from_uri(f"{base_url}/api-schema", headers=headers)

APIWorkflow = schema.as_state_machine()
TestAPI = APIWorkflow.TestCase

@pytest.fixture(scope="session")
def authorization_header():
return headers

@schema.parametrize(endpoint="/profile")
@settings(max_examples=1)
def test_api_post(case, authorization_header):
case.headers = authorization_header
response = case.call()
case.validate_response(response)

AND

following way:

    @pytest.fixture(scope="session")
    def rest_schema():
        return schema

@pytest.fixture
def state_machine(rest_schema, authorization_header):
class APIWorkflow(rest_schema.as_state_machine()):
def before_call(self, case):
case.headers = {}
case.headers = authorization_header

        def validate_response(self, response, case):
            case.validate_response(response)
    
    return APIWorkflow

def test_statefully(state_machine):
state_machine.run(
settings=settings(
max_examples=1,
deadline=None,
stateful_step_count=1,
)
)

I have also looked up the discussion regarding stateful testing here, #https://github.com/schemathesis/schemathesis/discussions/1212

What I expected: I expected schemathesis to do the POST /profile operation and pick the #id attribute from response and fill that in for /profile/{id} GET rest call.

What I got: Now the problem it does not pick the link from POST method, it just tries to call it individually. It just ignores the response from /profile POST operation and performs the GET /profile/{id} operation using a random value for {id}.

My questions:

What am I missing here to link the GET profile/{id} endpoint to connect with the POST /profile operation? After fixing 1): Could we also use these links for automated testing of DELETE operations, where the DELETE operation would be /profile/{id} ?
Environment information:
Python version: 3.9
Operating System: Mac OS Monterey 12.0.1
Django version: 4.1.1
Rest framework version: 3.14.0
Schemathesis version: 3.23.0 \

0

There are 0 answers