How To Work With Magic Numbers in Linux?

H

Okay, peeps gear up your shoes, as today we are going to touch a very interesting and undiscovered topic. The main objective to publish this article is to throw some light on magic numbers and file headers. We are going to take some questions like;

  • How to extract the file on the basis of magic numbers?
  • How to corrupt file on magic numbers?
  • How to repair file in the context of magic numbers in Linux environment?

Magic Numbers

Magic numbers are the first few bytes of a file which are always unique for a particular file. These unique bytes are referred to as magic numbers and sometimes referred to as file signatures. These are bytes are mainly used by the system to differentiate between and recognize different files without using the file extension.

How to locate Magic Numbers in File Signature?

Mostly files have signature byte at the beginning of a file, but some file systems have a signature in the offset parts. For instance, file system ext2/ext3 have bytes 0x53 and 0xEF at the 1080th and 1081st position.

  • However, some files don’t have magic numbers like a plain text file, but they can be identified using the character set. The file signature can be checked using this command;

file -i *name_of_file*

  • Magic numbers are not usually visible to the users, but they can easily view it using a hex editor or xxd command.
  • Changing these bytes can corrupt the file as most of the tools can’t recognize the file after making changes.
  • The file command in Linux readers reads the file signature before reading it.
  • Let’s take an example of a PNG file to better understand the magic numbers. You can view the hex of a file by typing the following command in a Linux terminal.

xxd image.png | head

This command will result in the first set of file bytes;

89 50 4e 47 0d 0a 1a 0a
// magic number of PNG file

These numbers help the system in identifying the type of file being used. Files which are not written with the help of these numbers can be identified using magic numbers.

Similarly to this example, the zip file example can be studied.

xxd test.zip | head

The result of this command will be;

50 3b 03 04
// magic number of zip file

Appending One File to another and identifying the division with Magic numbers

This operation can be easily performed with the help of python. First, we will read the bytes of two numbers and write them one by one to another empty file. In this article, we will combine a PNG with a Zip file.

“””
The first two lines open the two files to be read byte by byte
The third line opens an output file to be written to byte by byte
“””

input_file_1 = open(“image.png”, ‘rb’).read()
input_file_2 = open(“test.zip”, ‘rb’).read()
output_file = open(“output.png” , ‘wb’)

output_file.write(input_file_1)
output_file.write(input_file_2)

With this python code, we can obtain output.png, on the command;

xxd output.png | head

You can notice in this file that it begins with the same 8950 4e47 0d0a 1a0a hex. However, if we run the command;

xxd output.png | grep “PK”

This will search for magic numbers of zip file amongst the hex.

Well, we have so much more to share with you on this topic which we will disclose in our second part of the article. So, keep reading valuable posts.

About the author

Arpit Agarwal

I am a freelancer content writer, web developer and Video editor who loves to write technical stuff and on the other hand makes awesome videos as well. I like to make people happy with my writing and also try to make sure, you come back to read more.

By Arpit Agarwal

Most common tags

%d bloggers like this: