Repository navigation
Do not abort connection after encoding error in postgresql - #537
Conversation
Though they are failling right now.
This workflow is the most complicated, because database returns `n` rows and then waits for another `Execute` packet. Test that "encoding error" will not block, abort or desynchronize the connection.
| self.assertEqual(row, None) | ||
|
|
||
| loop = asyncio.get_event_loop() | ||
| loop.run_until_complete(test()) |
There was a problem hiding this comment.
I fiound that python has high-level helper asyncio.run for that. Lets start to use it?) Works since python3.7.
There was a problem hiding this comment.
I wanted to use that, because I use it in small scripts myself. But I've read from the docs:
This function cannot be called when another asyncio event loop is running in the same thread.
I don't know whether our test run in parallel, but if they do, this can cause a problem (or I just don't understand this statement correctly)
There was a problem hiding this comment.
then lets create new event loop for that?) use new_event_loop instead of get_event_loop
There was a problem hiding this comment.
Okay, but I don't quite understand the reasons. Could you explain why?
There was a problem hiding this comment.
it will help run code in parallel if need. every thread or test case will have own loop and it will safe to stop loop on failure and don't touch other tests that can rely on common loop in current thread
| self.assertEqual(ex.exception.message, | ||
| 'encoding error in column "value_bytes"') | ||
| row = await conn.fetchrow(select_query, data['id']) | ||
| self.assertEqual(row, None) |
There was a problem hiding this comment.
hm, can you explain why we expect None here? As I understand:
- we inserted data and it is already in the database
- We tried to get this data in the transaction, took error from acra side and handled exception
- We try to get same data but get None as result, not exception due to same error and not error that
selectreturns no data?
There was a problem hiding this comment.
You are a little wrong, here is the correct workflow:
- We insert corrupted data that will trigger decryption error and so it is accessible for the next operations.
- We begin transaction and insert another row of data.
- We trigger a decryption error and expect that db-driver does a rollback
- We check that indeed, because of the rollback, the data that we insert in the transaction, has not been saved.
There was a problem hiding this comment.
oh, thanks. can you add some comment above this check with explanation why we expect here None?
Because I'm sure that someone will forget that most of ORM use ROLLBACK on any exception. Or mention that after that we expect that ORM will do ROLLBACK. Just for case if event this ORM will change his logic and stop do ROLLBACK on errors.
...instead of explicitly capturing the exception and then hacking around to find out which of the properties contains it to test that it's the same as we expected. It also allows not caring about '\n'.
This PR implement skipping postgres packets from database after encoding error, almost in the same way as the previous PR does for mysql. It also adds a bunch of tests for different cases:
npackets and waits for the next request)The only issue with postgres is that a state of transaction is stored at database side, and changes in case of some errors (even parsing ones). When the state is changed,
commitis basicallyrollback:But our error will not trigger that:
So, for now I will mention it in the docs. Currently there is no straight and simple solution.
P.S.
test.pycrossed the milestone of 10 thousands lines, so my congratulations! It takes 5 second for my formatter to proceed the file. That's what I call success.Checklist
with new changes
CHANGELOG.md is updated (in case of notable or breaking changes)CHANGELOG_DEV.md is updatedBenchmark results are attached (if applicable)Example projects and code samples are up-to-date (in case of API changes)