February 2023 – Spencer Heckathorn

It Starts Somewhere I wish I had tracked some of these things a little closer. I did start a blog post in 2019, but I know it was months, if not longer after I started messing with AI deep fakes. I dipped my toes into the waters with DeepFaceLab the results were amazing, but the process was painstaking. You had to take a video, cut it up by frames, and then run the program. Mostly it was amazing, but I had issues along the way, such that I only made a few throwaway pieces of content that I don't even think I bothered to share with friends. I was still impressed and hooked. I got a copy of Neural Networks from Scratch in Python, which I'm slowly working through. I don't know if my suggested edits made it to the final copy, but I suggested an edit or two. Things Get Spicy Then sometime in mid-2021, there was Shiloh and Tim... They were in love, and Shiloh ended up dead, but don't worry because Tim was a firetruck, and it turns out Shiloh was always dead as it turns out she was really an evil magician the entire time. The story gets way too inappropriate for this blog, but you can thank EleutherAI - https://6b.eleuther.ai/ Which, for the most part, seems to be down at this time. However, the output was fun and exciting, much better than GPT2. I used this AI to develop interesting concepts and test what kinds of knowledge it might have locked away. For the most part, I came away with the sense that people do fall into predictable patterns. I've been dabbling in AI stuff on the side for a few years now. Several years ago, I ran GPT2 on my machine to produce Shakespeare-like text using GPT-2-Simple - https://github.com/minimaxir/gpt-2-simple GPT2 was very easy for a human to detect something was off. Enter Game Changer Things progressed slowly as I was improving my 3D printing skills and working a ton, but at some point during 2021, I applied for the OpenAI Beta to GPT-3 and Codex (both or separately, I can't remember). I got accepted into the OpenAI Beta on 09/09/2021 and immediately got to work, seeing what I could get it to do. Codex turned out to be somewhat of a bust, and I think I know how better to use it now, but I haven't tried my new ideas yet. The code output was questionable at best and terrible most of the time, but GPT-3 was rather impressive. I didn't save much from the hours of playing around, just interesting prompts I could use to get interesting responses. A lot of work went into figuring out that formatting and spelling actually made a significant difference. The thing that was interesting about GPT-3 was the potential quality that I could get out of it with a little coaxing. Shortly after, I joined a platform named ShortlyAI (pun totally intended). I knew they were using GPT-3, but I wanted to see what they were doing to get more interesting outputs. Before I go on here, I want to point out that I started using Grammarly several years ago. During the normal course of using Grammarly and seeing myself make the same mistakes over and over, I would slowly get better without much effort on my part, I just needed to write, and things would work themselves out. I saw the same thing happen 10 or more years earlier when, as a person with great difficulty spelling things, I would get better and better at spelling things if I made myself go back and retype the word when it got marked as misspelled. Using a tool like ShortlyAI or GPT-3 is the same interesting interaction on another level, like upgrading from Word spell checker to Grammarly. I currently don't use ShortlyAI, but if you can swing $80 /mo, I would say it offers the most compared to the other paid writing tools. I recommend you check out Chibi AI and Chad's channel, which used to be AI Content Dojo. He can teach you tons about using ShortlyAI to great effect. Chad also promotes the idea of using the AI as a writing partner and writing with the AI instead of having it write for you. Back to the info dump! Images Oh My During 2022 I found myself very busy with work and trying to catch up on things. We had to move unexpectedly at the start of the year, and I hate to admit how long it took me to unpack... (I mean, I still haven't finished) But I still found a little time to start messing around with Stable Diffusion around the end of August or the start of September, and it was FUN! It still is fun. Leading up to this, of course, I was trying to get access to Dall-E 2, but I'm happy that instead, I had to figure out the ins and outs of Stable Diffusion. It brings to mind the importance of doing AI stuffs locally on your own machine when possible. For a very short few weeks brought me back around to GTP-2, and I eventually decided that was not the right choice. Current Resources Currently, I'm still very interested in running what I can locally, but I don't have the hardware to do so. Here is a list of things I'm looking at and additional information when it seems appropriate to add such. Keeping It Local GPT-J-6B (This is the same model Eleuther mentioned above is/was using): arrmansa/Basic-UI-for-GPT-J-6B-with-low-vram - Project for running GPT-J Locally Access and use GPT-J (GPT J ) | Towards Data Science - Article with some interesting information Internet Based Stuffs ChatGPT has been all the rage recently. I've left it out of this post because I've been working on another post covering tons of things I've done since the release. Here are a few related resources: f/awesome-chatgpt-prompts: This repo includes ChatGPT prompt - Prompts to get better results when using ChatGPT nicolaballotta/gpt3-wordpress-post-generator: - I haven't used this but plan to experiment with it later. Other This is everything else I wasn't entirely sure how to classify. (2) Update: Upgrading to 1.5B GPT-2, - Interesting Reddit post David Shapiro ~ AI - YouTube - David deserves his own post, but I'll do a quick rundown of his work here. Originally I found his channel on the OpenAI forums, and he had some very interesting approaches to making good prompts and doing interesting things with fine-tuning. Somewhere along the way, he started working on a cognition system for large language models. Watch this video about DIY ChatGPT with long-term memories to get an idea of where he is going, and then check out this video about "Dreaming," followed by this video announcing the Open Source version of the project. Here is a list of David's projects I want to highlight and save for later. For most of the projects, it asked that you fork the original project and not work off of daveshap's original. daveshap/LongtermChatExternalSources: GPT-3 chatbot with long-term memory and external sources (github.com) daveshap/GibberishDetector: Detecting gibberish as a type of sentiment analysis with GPT2 (github.com) daveshap/PythonGPT3Tutorial: Public Hello World to get used to Python and GPT-3 (github.com) daveshap/FinetuningTutorial: Finetuning tutorial for GPT-3 (github.com) daveshap/RAVEN_MVP_Public: Public MVP of Raven. It's been long enough, time to do a full send. (github.com) daveshap/MultiDocumentAnswering: Experiment to answer questions from arbitrary number of sources (github.com) daveshap/CreativeWritingCoach: Finetune a GPT-3 model to provide copy editing (prose) feedback and critique (github.com) These are things I need to investigate more: Introducing Whisper (openai.com) openai/whisper: Robust Speech Recognition via Large-Scale Weak Supervision (github.com) More to come later. I have started several ChatGPT posts and plan to make them into videos, but we will have to wait and see about them. PS - I recently tried and loved EmpireMediaScience/A1111-Web-UI-Installer if you are not technical, you can at least use this to get Stable Diffusion running on many machines (you still likely need 8 GB of VRAM). The featured image of this post is will be the favorite I've produced in the last couple of weeks using A1111. *"will be" is because I built something with ChatGPT to help me pick, but there are thousands of images, so for my next post, I will use my new tool as I write about it.