If you have json.loads(response.text)
in a Python script using requests
to perform a loop over API calls structured something like response = requests.get('https://example.com')
inside the loop, and you sporadically see an error stack ending in raw_decode
mentioning ValueError
from None
like this, you might have an off-by-one error in your loop condition:
`File ".../json/__init__.py", line ..., in loads
return _default_decoder.decode(s)
File ".../json/decoder.py", line ..., in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File ".../json/decoder.py", line ..., in raw_decode
raise ValueError(errmsg("Expecting value", s, err.value)) from None
Check to see if you have an off-by-one error that's making you enter the loop one more time than is actually necessary to fetch all the data.
- It's possible that you made a typo and that you just need to fix the condition for continuing your loop.
- It's also possible the at the API is lying to you and saying there's a "next" page when there isn't really one. In that case, consider adding an extra guard to the loop entry condition that verifies for example,
and response is not None
.
You'll often see this problem sporadically -- e.g. if the API's pagination is in multiples of 100, only when there happen to be exactly 800 records in the dataset (or maybe 799 or 801), with your code otherwise working.