I'm awestruck. I feed an AI all my personal notes and favorite books

[Lucasxp64]

notebooklm.google.com

I used the podcast generator from this Google AI tool, I extracted all of my notes from my Obsidian vault into a single .txt file, and I feed the AI it as a source, alongside some of my favorite books, and I prompted the podcast to be about speaking my personal struggles from the perspective from the books, and the whole podcast got fine-tuned to reading my notes based on those books mindsets.

This is a completely new step above merely chatting with an AI. I always dreamed of being able to actually feed my content into it and getting insights beyond just surface level garbage done with simple vector based techniques.

It's completely worth it. Although, you should mind your privacy if you're concerned.

It's incredible, I can now relate knowledge of books with my own personal struggles, ways of thinking and biases, it has massive context about me.

Now I should also automatically transcribe the dozens of audio journaling audio recordings I had as well. Those are really hard to go through. If I feed them into the AI it will be wonderful to extract insights and common patterns from them.

[Leo Gura]

I heard about this just yesterday and wanted to try it out.

[Ves]

Is Notebook LM really so good? My one experience with it was feeding it a paper in pure mathematics straight from the arXiv, and I got the vaguest overview that you could possibly imagine. Sure, it was an extremely technical paper and there are limits to what I should expect. Maybe the system can't even parse such a paper very well.

At the same time... the podcast was basically just two people getting sidetracked constantly, and when they were on topic, they could only make up incredibly banal analogies. Imagine two people with ADHD adlibbing a podcast over a topic they have zero knowledge about. It was even more concerning because I first heard about Notebook LM from other graduate students (most of them in some kind of science field) who seemed to find it valuable.

It is better with less technical data? They can speak meaningfully and productively in a focused way? Maybe I should try it again with excerpts from a novel to see how nuch better it is.

[integral]

@Leo Gura also check out Gemini deep research.

[Whitney Edwards]

@integral can we feed our journals into gemini and get insights from it? 

[integral]

10 minutes ago, Whitney Edwards said:

@integral can we feed our journals into gemini and get insights from it? 

Yes, but right now you would have to copy paste each page into the notebook. It might take a while seeing as you write a book a day. 😊

[Whitney Edwards]

Just now, integral said:

Yes, but right now you would have to copy paste each page into the notebook. It might take a while seeing as you write a book a day. 😊

I see.

[Lucasxp64]

11 hours ago, Ves said:

It is better with less technical data? They can speak meaningfully and productively in a focused way? Maybe I should try it again with excerpts from a novel to see how nuch better it is.

Try to write the "Custom" prompt for the podcast. But yeah, they keep getting sidetracked and are ADHD, the results vary a lot depending on the prompt, but that's what I wanted, I wanted them to be spit-balling based on my journal. Improve your prompt if they are getting too sidetracked.

For extremely specific answers, you should not use the podcast. The podcast is not an instruction model. I tried to give specific instructions, they don't follow them.

You should use the CHAT for specific/technical answers and questions, it even provides the sources/specific locations in the files.

I'd be careful with uploading PDF formats of Mathematics, Chemistry, etc, anything with a lot of notations or complex syntax. Unless they are in something like LaTeX already, like all other models out there. Maybe it's good at parsing them?

[Lucasxp64]

 

9 hours ago, Whitney Edwards said:

@integral can we feed our journals into gemini and get insights from it? 

I used a script that concatenates all my markdown files from my Obsidian vault into a single .txt file.

If it's a handwritten notebook, well, I guess you could scan and feed some other AI with the best quality OCR you can get, and feed the AI the outputs. Perhaps it can already OCR if you upload it as PDF? Someone can confirm this?

Also, I saw that it actually accepts audio files as well, so I can just upload my voice notes too.

But I might hit the upload limit and I might need to circumvent it by transcribing the audios with Whisper, and concatenating the output text as a single file. Or concatenating the audiofiles into a single file using FFMpeg. There must be a setting to do that without re-encoding so it would be fast.

Right now, It has a limit of 50 files, and "Each source can contain up to 500,000 words, or up to 200MB for uploaded files." 

https://support.google.com/notebooklm/answer/14276468?hl=en

With OneNote, I tried to export my old Notebooks as PDF, but it glitched. I might try again with some other exporter format or software. As long as I can get the raw text extracted from there. Should be similar for any journaling software, some exporter. Choose to export to MARKDOWN format if possible, that's the native format all LLMs use, but it's not that important, they can make sense of garbled formatting.

To upload your "EPUB" files of books, you I used Calibri to convet them to .TXT files.

It would be trivial to make a script that takes even a whole folder of files and concatenate/Splits them as needed for importing there. (If it goes past limits of 500k words, or limit per file, etc)

 

 

Edited Tuesday at 12:24 PM by Lucasxp64