Zoom Logo

TW presents: What if your databases never forgot - A story of time travel - Shared screen with gallery view
Charlotte Oetken
22:48
CoC: https://www.thoughtworks.com/code-of-conduct-germany
Jeremy Taylor
23:49
Hi :)
Timo
24:10
Hi
Charlotte Oetken
27:17
Please find our CoC here: https://www.thoughtworks.com/code-of-conduct-germany
Charlotte Oetken
27:43
Feel free to ping your questions to the chat, we will collect them and they will be answered at the end of the talk :)
Kiriakos
29:42
Sound ok on you end?
Kiriakos
29:47
your
Luke
29:59
sounds fine here
Hitesh Garg
30:00
Good on my side
Daniel
30:06
all good
Charlotte Oetken
32:40
When you are having issues with your sound, try to adjust your settings by clicking on the arrow next to the microphone. There you can find the audio settings
Susanne Kirndorfer / ThoughtWorks (she/her)
36:03
Feel free to ping your questions to the chat, we will collect them and they will be answered at the end of the talk :)
dmytri
40:02
Ex-ThoughtWorker here, currently working on a append-only database with git-llike collaboration features called TerminusDB
dmytri
48:59
our query language is also a datalog :)
Manuela
49:44
How is it Performing /Scalable, especially with high record Counts (Millions) with lots of changes on them?
Manuela
51:13
How to handle the typical GDPR usecases?
Pawarit Laosunthara
55:24
So time-travel is strictly limited by the retention period of your Transaction Log (e.g. Kafka)?
Alex
59:52
How do you handle data structure migration? Does the application then need to be able to remind all possible data structures a document might have had over time, e.g. maybe town was renamed to city at some point in time. Or do you time travel the code that handles the data structure as well :)
Christoph Deil
01:02:20
What would you use with PostgreSQL to support time travel? Use temporal_tables extension, or just a custom implementation with extra temporal columns or tables?
Manuela
01:02:25
Ist regarding the General Performance of the database
Jakub Milkiewicz
01:02:38
can you elaborate more on “detachment of query and storage,” does it mean that data is stored within application (let’s say in heap space )?
Manuela
01:02:39
whether it Drops with many entries
Moritz Lawitschka
01:03:21
Have you considered using AWS DynamoDB for a datalog since it supports append-only tables? Do you have any experience to share?
dmytri
01:04:04
Would love to comment on the question from the point of view of TerminusDB if anyone is interested
Harman Bains
01:05:11
What is the nature/schema of the underlying graph implementation for Crux? RDF/Triple Store? Something Crux specific?
Timo
01:05:58
Same for Datahike
dmytri
01:06:22
@harman, terminusdb is rdf/tripple based
MichaelNietzold@Cologne
01:10:00
i read about an other tactics for GDPR/DSGVO: encode each transactions and each data with a private key per customer (saved on different place then the data), then in case when deleting: just delete the private key of this customer and then the data is not anymore decodable - but i don't know how performant is this tactics - but then you don't have to delete backups and transactions logs
dmytri
01:11:28
encoding transactions is a great approach
Harman Bains
01:13:55
@MichaelNiet flureeDB is a graph database with built in cryptography features and they use the key approach to deal with GDPR
Timo
01:13:58
You can use Datahike with a Dynamo backend:)
James Henderson
01:14:46
In Crux, we split transactions into the transaction log and document store. The transaction log doesn't contain any personally identifiable information, just references to documents in the document store. This means that, on evict, we still don't need to change the transaction log. More details here: https://juxt.pro/blog/crux-doc-store#_separating_transactions_from_documents
Harman Bains
01:15:41
Awesome thank you!
Jakub Milkiewicz
01:15:45
given that i have a number of instances of my application , does it mean that each of these instances will have its own index (probably based on RocksDB)? How do you provide consistency that some “writes” would be visible “faster” to where they were executed (to which instance) before other instances?
Rahul De
01:16:49
Slides: https://speakerdeck.com/lispyclouds/what-if-your-databases-never-forgot
Timo
01:17:56
Thanks Rahul!
Manuel Huppertz
01:18:00
Thanks!
Christoph Schubert
01:18:01
Thanks Rahul
Manuela
01:18:01
Thanks all of you!
Timo
01:18:02
Bye
Daniel
01:18:02
Thanks for the event!
Jorge Florian
01:18:03
Thanks Rahul!
Deniz Saner
01:18:04
thanks!
Oliver
01:18:05
Thanks for this interesting talk
Kiriakos
01:18:06
Thanks guys!
Christoph Schubert
01:18:06
Very useful!
Michael P
01:18:06
Thanks a lot!!
Boris
01:18:07
Thanks! Bye!
Deniz Saner
01:18:08
bye ✋🏻
Mateus Zitelli
01:18:10
thanks!!
Merlin Thomas
01:18:11
thanks!
jenisys
01:18:14
thanks
Sergii Tretiak
01:18:17
thanks!
Malcolm Sparks
01:18:22
Goodbye!
Simon Bäumler
01:18:22
Great talk! Thank you very much! :)
Søren Sjørup
01:18:24
thx
Henrik M
01:18:26
thanks from Denmark 🙂