LZMA SDK Compress Decompress

7z is one of the best if not the best file compression available. Best of all, it is open source. The engine behind it is the LZMA compression method. I was integrating the sdk (https://www.7-zip.org/sdk.html) in my project, but however, can't get a quick start on the compress decompress process. After searching the internet, I figured out on a surface level how it all works. Partially thanks to the question in https://stackoverflow.com/questions/7646328/how-to-use-the-7z-sdk-to-compress-and-decompress-a-file. So, below, I write down my basic understanding.

Compress

Basically, the compressed file will contain 3 things with the first 2 are metadata:

1. The first 5 bytes are compression properties
2. The next 8 bytes are file size before compression
3. The compressed bytes

var encoder = New Encoder();
encoder.WriteCoderProperties(outStream); // Write properties
encoder.Write(BitConverter.GetBytes(inputFileSize), 0, 8); // Write uncompressed file size
encoder.Code(inStream, outSteam, inStream.Length, -1, null); // Actual compress

Decompress

To decompress the file, the metadata needs to be provided to the decoder. My code initially threw an error because there is no metadata.

var properties = new byte[5];
inStream.Read(properties, 0, 5); // Read properties

var fileSizeBytes = new byte[8];
inStream.Read(fileSizeBytes, 0, 8); // Read uncompressed file size
var fileSize = BitConverter.ToInt64(fileSizeBytes, 0);

var decoder = New Decoder();
decoder.SetDecoderProperties(properties); // Provide the properties to decoder
decoder.Code(inStream, outStream, inStream.Length, fileSize, null); // Actual decompress

Read-only File System Error in Linux

I was moving the content of CentOS boot drive to a new hard drive. CentOS has MBR partition with xfs file system. It worked great, boot fine, but when I tried to do yum install, it barked that it can't do the install because the file system is read-only.

After a decent amount of research, I found out that the problem lies on the /etc/fstab. Because it is new hard drive, the UUID is different and grub2-mkconfig used the new UUID to configure the grub.cfg. However, when it is booted, it checked the /etc/fstab and found the discrepancy. Once I changed the /etc/fstab to reflect the new UUID, the error went away.

Understanding AWS Security Group

We were playing with AWS Aurora Serverless. After figuring out how to configure and start the database cluster, we were having trouble connecting to it from our EC2 instance. Few hours later, I tried tracing what was going on with Flow Logs in the subnet. We realized it was a network and/or security issue. Checking all possible connections and security, we think everything has been configured correctly. But only after trial and error, we found out that our understanding of security group was incorrect.

Due to AWS security group being promoted as stateful, we understand it as if we specify an entry in Inbound tab, we don't need to specify it in Outbound. Sadly, that is not what stateful means. Each entry specifies the allowed origin of the network request and the response will be automatically allowed. For example, if we allow in Inbound only, a request can come from outside and allowed to the EC2 instance and out, but a request from the instance won't be allowed to go out of the instance. And that is precisely what our problem is, we have an entry in the Inbound tab, but we try to connect from inside the EC2 instance, but was rejected because we don't allow the connection on the Outbound tab. As soon as we allow the connection on the Outbound tab, the problem is fixed.