March 2023 – Spencer Heckathorn

Today I will look at some techniques and examples in Python for embedding data in video frames. This idea came to me after Congress grilled the CEO of TikTok for 5 hours regarding the privacy of Americans and their use of TikTok. Me, I love TikTok. I would switch over if an American-based app were doing the same things. However, everyone is falling short of delivering the same experience in connection and exposure to new things as TikTok does. I've learned many new things and had mini-viral videos here and there. Anyhow, as most of our members of Congress embarrassed the rest of us with a first graders' understanding of technology and apps, a few had good points and questions. It got me wondering, "if I were an international company looking to steal data, how would I do it without letting anyone know?" Ideally, I would want to create a data stream that could exfiltrate data without anyone realizing that data was moving. I could encrypt the data and ensure that if anyone found my streams, they would not know what I was streaming. Still, having a bunch of extra data streams unaccounted for is risky, but what if I hid the data in plain site? Enter Steganography If you don't know comes from the Greek words "steganos," meaning "covered" or "concealed," and "graphein," meaning "to write," and it is the practice of concealing messages or data within other non-secret data. Personally, I'm too lazy to do this, although I should. With the proper usage, you could hide data in images, audio, video (as I will show you a possible method here), or potentially within network packets. Using such techniques could aid bypassing censorship, protecting intellectual property, or just helping hide your grandmother's recipes. However, in this case, if developers of an app are so inclined or pressured into doing so, they could hide things such as depth data, location data, "pre-frames," cut frames, and more inside the videos posted on the platform and would only need to access the videos to extra the extra data which could be encrypted. All of this can be done by manipulating pixel values inside the video as posted. Interestingly enough, as framerates increase and display density increases, it becomes easier to hide more data. Continuing with our thought experiment you might be able to imagine how a country such as China would be able to access all kinds of data in an app for sharing videos while being able to claim they don't leave the company that owns and controls the app able to say they also innocent. Now, I want to point out this is all potential, and I have no evidence that something like this is happening. Again I love TikTok and hope it sticks around, but I want to understand better why our US government might want to ban such an app. Let's Look at Sample Code The following is not tested yet. I used Github Copilot to help me write these samples and have not tested them yet (that will be for the next post). Later, I will test and correct this code and produce some samples. import cv2 import numpy as np from bitarray import bitarray # Load the video file video_file = 'path/to/your/video/file.mp4' video = cv2.VideoCapture(video_file) # Check if the video file was loaded successfully if not video.isOpened(): print("Error: Could not open the video file.") exit() The first section is opening our video file. This next section will set a "message" variable to anything we want. In this case, it says "This is a secret message." we could replace that with photo depth data, location data, or any arbitrary data we want. We might hit size limits, but we are not worried about that now. # The secret message we want to hide message = "This is a secret message." # Convert the message to binary message_binary = bitarray() message_binary.frombytes(message.encode('utf-8')) This next part should modify the blue pixel data. If you don't know, encoding images and video, or even audio for that matter, uses a principle of least significant bit (LSB) to eliminate data from these file types allowing us to shrink the size by getting rid of the things we either can't see or hear. In this case, we will use the blue channel, which is the least noticeable to the human eye. In my original ideation, I thought you could likely story extra blue data in the darkest areas of the images or frames on the video. This is an older idea of mine as well. Back in the day, when you had to make small videos, they had artifacts so you could send them over email for a good laugh. So I wondered if data about who encoded the video could be hidden in the large pixels. I think the idea is still valid. As you can see, with some video codecs, pixelation still occurs in certain dark shots. However, with the current technology, you don't need to have noticeable pixelation at all. This random video I found from a long time ago shows some pixelation: (2) Rep Don Young Urges Producing American Energy & Opening ANWR - YouTube I'm by no means saying this is what is happening in this video, but my point is that, at one time, this type of video quality was acceptable. Back to the code: # Define the output video file output_video_file = 'path/to/your/output/video/file.mp4' fourcc = cv2.VideoWriter_fourcc(*'mp4v') frame_size = (int(video.get(cv2.CAP_PROP_FRAME_WIDTH)), int(video.get(cv2.CAP_PROP_FRAME_HEIGHT))) fps = int(video.get(cv2.CAP_PROP_FPS)) output_video = cv2.VideoWriter(output_video_file, fourcc, fps, frame_size, isColor=True) # Embed the data in the video frames frame_idx = 0 bit_idx = 0 while video.isOpened(): ret, frame = video.read() if not ret: break # Embed the data in the frame for i in range(frame.shape[0]): for j in range(frame.shape[1]): # Modify the pixel's blue channel to store the secret data blue_channel = frame[i, j, 0] if bit_idx < len(message_binary): frame[i, j, 0] = (blue_channel & ~1) | message_binary[bit_idx] bit_idx += 1 else: frame[i, j, 0] = (blue_channel & ~1) # Write the modified frame to the output video output_video.write(frame) frame_idx += 1 # Release the video files video.release() output_video.release() We are creating an output video with the same properties as the original video (codec, frame size, and FPS). Then we loop through the video frame by frame, modifying the blue channel of each pixel based on the binary message. Again this code is pseudo-code, but for my next post, when I find some time, I'll make a working demo and hide some data in a video. I want to dive even deeper into this and see what kind of data we could get back as a phone app developer, but that is not something I plan on doing anytime soon, so if you happen to know something about that, please let me know by commenting.