Forensics

This post (Work in Progress) lists the tips and tricks while doing Forensics challenges during various CTF’s.

This might be a good reference Useful tools for CTF

File Formats

There could be CTF challenges in which the header format is wrong and the file won’t open as expected.

Hex File Header and ASCII Equivalent

File headers are used to identify a file by examining the first 4 or 5 bytes of its hexadecimal content. Taken from Hex file and Regex Cheat Sheet and Gary Kessler File Signature Table is a good reference for file signatures.

Filetype       Start             Start ASCII Translation

  ani         52 49 46 46                                 RIFF
  au          2E 73 6E 64                                 snd
  bmp         42 4D F8 A9                                 BM
  bmp         42 4D 62 25                                 BMp%
  bmp         42 4D 76 03                                 BMv
  cab         4D 53 43 46                                 MSCF
  dll         4D 5A 90 00                                 MZ
  Excel       D0 CF 11 E0
  exe         4D 5A 50 00                                 MZP (inno)
  exe         4D 5A 90 00                                 MZ
  flv         46 4C 56 01                                 FLV
  gif         47 49 46 38 39 61                           GIF89a
  gif         47 49 46 38 37 61                           GIF87a
  gz          1F 8B 08 08
  ico         00 00 01 00
  jpeg        FF D8 FF E1
  jpeg        FF D8 FF E0                                 JFIF
  jpeg        FF D8 FF FE                                 JFIF
  Linux bin   7F 45 4C 46                                  ELF
  png         89 50 4E 47                                  PNG
  msi         D0 CF 11 E0
  mp3         49 44 33 2E                                  ID3
  mp3         49 44 33 03                                  ID3
  OFT         4F 46 54 32                                  OFT2
  PPT         D0 CF 11 E0
  PDF         25 50 44 46                                  %PDF
  rar         52 61 72 21                                  Rar!
  sfw         43 57 53 06/08                               cws
  tar         1F 8B 08 00
  tgz         1F 9D 90 70
  Word        D0 CF 11 E0
  wmv         30 26 B2 75
  zip         50 4B 03 04                                   PK
  wav         52 49 46 46 ?? ?? ?? ?? 57 41 56 45

Correct Headers: SQLite3

0000000: 5351 4c69 7465 2066 6f72 6d61 7420 3300  SQLite format 3.
0000010: 0400 0101 0040 2020 0000 000b 0000 000b  .....@  ........
0000020: 0000 0000 0000 0000 0000 0002 0000 0004  ................

Kaitai Struct

Apart from the above reference, use Kaitai Struct, Kaitai Struct Visualizer to verify the header.

Usage: ksv [options] <file_to_parse.bin> <format.ksy>...

  -I, --import-path [DIRECTORIES]  .ksy library search path(s) for imports (see also KSPATH env variable)
      --opaque-types [BOOLEAN]     opaque types allowed, default: false
      --version                    show versions of ksv and ksc

Example:

ksv tunn3l_v1s10n.bmp ~/utils/forensics/kaitai_struct_formats/image/bmp.ksy

[-] [root]                               00000000: 42 4d 8e 26 2c 00 00 00 00 00 ba d0 00 00 ba d0 | BM.&,...........
[-] file_hdr                           00000010: 00 00 6e 04 00 00 32 01 00 00 01 00 18 00 00 00 | ..n...2.........
  [.] file_type = 42 4d                00000020: 00 00 58 26 2c 00 25 16 00 00 25 16 00 00 00 00 | ..X&,.%...%.....
  [.] len_file = 2893454               00000030: 00 00 00 00 00 00 23 1a 17 27 1e 1b 29 20 1d 2a | ......#..'..) .*
  [.] reserved1 = 0                    00000040: 21 1e 26 1d 1a 31 28 25 35 2c 29 33 2a 27 38 2f | !.&..1(%5,)3*'8/
  [.] reserved2 = 0                    00000050: 2c 2f 26 23 33 2a 26 2d 24 20 3b 32 2e 32 29 25 | ,/&#3*&-$ ;2.2)%
  [.] ofs_bitmap = 53434               00000060: 30 27 23 33 2a 26 38 2c 28 36 2b 27 39 2d 2b 2f | 0'#3*&8,(6+'9-+/
[-] dib_info                           00000070: 26 23 1d 12 0e 23 17 11 29 16 0e 55 3d 31 97 76 | &#...#..)..U=1.v
  [.] len_header = 53434               00000080: 66 8b 66 52 99 6d 56 9e 70 58 9e 6f 54 9c 6f 54 | f.fR.mV.pX.oT.oT
  [?] color_mask_alpha                 00000090: ab 7e 63 ba 8c 6d bd 8a 69 c8 97 71 c1 93 71 c1 | .~c..m..i..q..q.
  [?] color_mask_blue                  000000a0: 97 74 c1 94 73 c0 93 72 c0 8f 6f bd 8e 6e ba 8d | .t..s..r..o..n..
  [?] color_mask_green                 000000b0: 6b b7 8d 6a b0 85 64 a0 74 55 a3 77 5a 98 6f 56 | k..j..d.tU.wZ.oV
  [?] color_mask_red                   000000c0: 76 52 3a 71 52 3d 6c 4f 40 6d 52 44 6e 53 49 77 | vR:qR=lO@mRDnSIw
  [?] is_color_mask_here               000000d0: 5e 54 53 39 33 70 58 52 76 61 59 73 5f 54 7e 6b | ^TS93pXRvaYs_T~k
  [?] is_color_mask_given              000000e0: 5e 86 74 63 7e 6a 59 76 62 50 76 5e 4c 7a 62 50 | ^.tc~jYvbPv^LzbP
  [?] color_mask_given                 000000f0: 87 6d 5d 83 69 59 8d 73 63 9b 81 71 9e 84 74 98 | .m].iY.sc..q..t.

BMP

BMP File Format
In the Header, look for DataOffset, it contains the Offset from beginning of file to the beginning of the bitmap data.
Check the file size that should match with the height and width of the image.
The pixel dimensions can be determined by multiplying both the width and the height by the dpi.
There could be cases where height and width of the image are wrong. In that case, check the size of the image and calculate the possible height and width. - Also check Bits per pixel. (Check if anything is hidden in Red/Blue/Green channel - Refer Stegsolv/CyberChef (RenderImage))
- If it’s 24 bit (RGB), we need to divide the filesize by 3 to determine the number of pixels. Based on that we can calculate the actual height and width of the image. Refer PicoCTF tunn3l_v1s10n
- If it’s 32 bit (RGBA).

PNG

The first eight bytes of a PNG file always contain the following (hex) values: 89 50 4e 47 0d 0a 1a 0a. This signature indicates that the remainder of the file contains a single PNG image, consisting of a series of chunks beginning with an IHDR chunk and ending with an IEND chunk. A valid PNG image must contain an IHDR chunk, one or more IDAT chunks, and an IEND chunk.

Chunk layout

Each chunk consists of four parts:

Length: A 4-byte unsigned integer giving the number of bytes in the chunk’s data field. The length counts only the data field, not itself, the chunk type code, or the CRC. Zero is a valid length.
Chunk Type: A 4-byte chunk type code. For convenience in description and in examining PNG files, type codes are restricted to consist of uppercase and lowercase ASCII letters (A-Z and a-z, or 65-90 and 97-122 decimal).
Chunk Data: The data bytes appropriate to the chunk type, if any. This field can be of zero length.
CRC: A 4-byte CRC (Cyclic Redundancy Check) calculated on the preceding bytes in the chunk, including the chunk type code and chunk data fields, but not including the length field. The CRC is always present, even for chunks containing no data.

Summary of chunks

This table summarizes some properties of the standard chunk types.

Critical chunks (must appear in this order, except PLTE is optional):

Name  Multiple  Ordering constraints
        OK?

IHDR    No      Must be first
PLTE    No      Before IDAT
IDAT    Yes     Multiple IDATs must be consecutive
IEND    No      Must be last

Ancillary chunks (need not appear in this order):

Name  Multiple  Ordering constraints
        OK?

cHRM    No      Before PLTE and IDAT
gAMA    No      Before PLTE and IDAT
iCCP    No      Before PLTE and IDAT
sBIT    No      Before PLTE and IDAT
sRGB    No      Before PLTE and IDAT
bKGD    No      After PLTE; before IDAT
hIST    No      After PLTE; before IDAT
tRNS    No      After PLTE; before IDAT
pHYs    No      Before IDAT
sPLT    Yes     Before IDAT
tIME    No      None
iTXt    Yes     None
tEXt    Yes     None
zTXt    Yes     None

Metadata

Metadata is data about data. Different types of files have different metadata. The metadata on a photo could include dates, camera information, GPS location, comments, etc. For music, it could include the title, author, track number and album.

Timestamps

Timestamps are data that indicate the time of certain events (MAC):

Modification : when a file was modified
Access : when a file or entries were read or accessed
Creation : when files or entries were created

Types of timestamps

Modified
Accessed
Created
Date Changed (MFT)
Filename Date Created (MFT)
Filename Date Modified (MFT)
Filename Date Accessed (MFT)
INDX Entry Date Created
INDX Entry Date Modified
INDX Entry Date Accessed
INDX Entry Date Changed

Timeline Patterns

Pattern: Run steghide Tool on File

If steghide tool was used to hide information in a file,
M=Date Changed(MFT)=INDX Entry Date Modified = INDX Entry Date Changed > A,C

Steganography

Images

If you are looking for hidden flag in an image first check with

exiftool

file, exiftool command, and make sure the extension is correctly displayed.

user could “add a comment” to an embedded thumbnail image using the command:
exiftool a.jpg -thumbnailimage -b | exiftool -comment=wow - | exiftool a.jpg -thumbnailimage'<=-'
The hidden flag could in base64 without ==.

strings

Sometimes, it is better to see lines only greater than x length.

strings RainingBlood.mp3 | awk 'length($0)>20' | sort -u

There is also srch_strings command part of sleuthkit that can display printable strings in files.

stegsolve

$ wget http://www.caesum.com/handbook/Stegsolve.jar -O stegsolve.jar
$ java -jar stegsolve.jar

CyberChef

Cyberchef also allows us to render image Multimedia -> Render Image and perform different functions.

Split color channels.

steghide

If there’s any text present in the Image/Wav file or the filename of the image or any link ( maybe to youtube video; video name can be the password ) that can be a passphrase to steghide. Sometimes, you may have to try all lowercase/ uppercase combinations.

steghide info filename.bmp
steghide extract -sf filename.bmp -v
steghide extract -sf message.wav -p hidden_stegosaurus

There might be a case where we have to bruteforce steghide with a wordlist. In that case, use stegcracker or stegseek.

binwalk/foremost

binwalk or foremost the file, just to make sure, there’s nothing extra stored in that image.
hexdump -C and look for interesting pattern may be? If you get 7z or PK they represent Zipped files. If so, you can extract those file with 7z x . If somehow, you get a passphrase for the image, then you might have to use steghide tool as it allows to hide data with a passphrase.
stegsolve: check all the planes. There’s a data-extracter, we may try to extract all the values of RGB and see if there’s any flag in that.
stegosuite

zsteg : detect stegano-hidden data in PNG & BMP
Mediaextract : Extracts media files (AVI, Ogg, Wave, PNG, …) that are embedded within other files.
Comparing two similar images to find the difference

compare hint.png stego100.png -compose src diff.png

Image Arithmetic We can do image addition, subtraction, multiplication, division, blending, logical AND/NAND, logical OR/NOR, logical XOR/XNOR, Invert/ Logical NOT, Bitshift Operators.
We can use gmic to perform XOR of the images.

gmic a.png b.png -blend xor -o result.png

JPEG : Jstego : program aims at providing a java solution to hide secret information(such as secret file) to JPEG images. Hiding algorithm contains Jsteg and F5. The main(probably the toughest) stuff is encoding and decoding JFIF files.
JPEG : Jsteg : jsteg is a package for hiding data inside jpeg files, a technique known as steganography. This is accomplished by copying each bit of the data into the least-significant bits of the image. The amount of data that can be hidden depends on the filesize of the jpeg; it takes about 10-14 bytes of jpeg to store each byte of the hidden data.
Repair Corrupted JPEG/JPG, GIF, TIFF, BMP, PNG or RAW Image

https://stylesuxx.github.io/steganography/

LSB Stegonagraphy

File are made of bytes. Each byte is composed of eight bits.

10101100

1st digit is MSB and Last digit is LSB

Changing the least-significant bit (LSB) doesn’t change the value very much.

10101100(base 2) == 172 (10)

changing the LSB from 0 to 1:

10101101(base 2) == 173 (10)

So we can modify the LSB without changing the file noticeably. By doing so, we can hide a message inside.

LSB Stegonagraphy in Images

LSB Stegonagraphy or Least Significant Bit Stegonagraphy is a method of stegonagraphy where data is recorded in the lowest bit of a byte.

Say an image has a pixel with an RGB value of (255, 255, 255), the bits of those RGB values will look like

1 1 1 1 1 1 1 1

By modifying the lowest, or least significant, bit, we can use the 1 bit space across every RGB value for every pixel to construct a message.

1 1 1 1 1 1 1 0

The reason stegonagraphy is hard to detect by sight is because a 1 bit difference in color is insignificant as seen below.

Color 1    Color 2
FFFFFE     FFFFFF

Decoding LSB steganography is exactly the same as encoding, but in reverse. For each byte, grab the LSB and add it to your decoded message. Once you’ve gone through each byte, convert all the LSBs you grabbed into text or a file.

There might be times, whether there is a message hidden using LSB encoding in a image in a particular way such as original.bmp, encoded.bmp. The encoded bmp might be X bytes of original image and then bytes are encoded. We have to check the patter in which flag has been written.

from pwn import *

with open("encoded.bmp", "rb") as file:
# 0x2d3 is the location from where the flag has been written
    file.seek(0x2d3)
    bin_str = ""
    # Assuming flag is of 50 characters. Each character is represented by 1 byte and can be from 0x00-0xFF and hence 50 * 8.
    for j in range(50 * 8):
        ## file.read reads a byte.
        # ord function returns the number representing the unicode code of a specified character.
        # Bit operator AND & get the LSB
        bin_str += str(ord(file.read(1)) & 1)

char_str = unbits(bin_str, endian = 'little')
print (char_str)

MSB Steganography

SigBits Steganography significant bits image decoder.
StegOnline: A web-based, enhanced and open-source port of StegSolve.

Powershell Steganography

Invoke-PSImage can encode a PowerShell script in the pixels of a PNG file and generates a oneliner to execute.

It can be decoded using Decode_PS_Stego. Read more details at Powershell Steganography.

QRCodes?

Install zbarimg

apt-get install zbar-tools

Usage

Read a QR-Code

zbarimg <imagefile>

Got a QR-Code in Binary 0101?, convert it into QR-Code by QR Code Generator

Images type

SVG

There could be text hidden inside the SVG file using tspan <https://developer.mozilla.org/en-US/docs/Web/SVG/Element/tspan>_ tag. The SVG <tspan> element defines a subtext within a <text> element or another <tspan> element. It allows for adjustment of the style and/or position of that subtext as needed.

For example:

<svg viewBox="0 0 240 40" xmlns="http://www.w3.org/2000/svg">
<style>
  text {font: italic 12px serif;}
  tspan {font: bold 10px sans-serif;fill: red;}
</style>
<text x="10" y="30" class="small">
  You are
  <tspan>not</tspan>
  a banana!
</text>
</svg>

PNG

pngcheck : pngcheck verifies the integrity of PNG, JNG and MNG files (by checking the internal 32-bit CRCs [checksums] and decompressing the image data); it can optionally dump almost all of the chunk-level information in the image in human-readable form.

Sound Files

Slow-Scan Television transmissions (SSTV)

It is easy to trasmit an image inside audio frequencies.

apt-get install qsstv
pactl load-module module-null-sink sink_name=virtual-cable
pavucontrol # A GUI will pop-up, go to the "Output Devices" tab to verify that you have the "Null Output" device
qsstv # The program GUI will pop-up, go to "Options" -> "Configuration" -> "Sound" and select the "PulseAudio" Audio Interface
# Back in the pavucontrol GUI, select the "Recording" tab and specify that QSSTV should capture audio from the Null Output

# click the "Play" button in QSSTV to start the receiver, and then play the audio file:
paplay -d virtual-cable message.wav

After the transmission has ended, cleanup by using the following commands:

pactl list short modules | grep null
22      module-null-sink        sink_name=virtual-cable
pactl unload-module 22

Spectrum Analysis

Open the file in Audacity or Spectrum Analyzer and probably analyze the Spectogram

Arrow next to the track name to switch from waveform (top) to logarithmic spectrogram (bottom).

Morse code possible? As all the morse data appears to be below 100 Hz, we can use a low pass filter (effects menu, cutoff 100 Hz) to ease transcription

Golang mp3 Frame Parser

In Audacity, we can zoom in to see any interesting patterns. View > Zoom in

Amplitude?

If we are provided a wave file, it can interesting to see the amplitudes. Check the unique amplitudes? See if the number of unique amplitude is equal to 16 (hex). In that case, probably the amplitude first two digits may represent hex? For example, if array([10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85]), 10 might represent 0x0 and so on.

from scipy.io import wavfile
# Return the sample rate (in samples/sec) and data from an LPCM WAV file.
samplerate, data = wavfile.read(filename, mmap=False)

Patterns

If you find a pattern like below

it might mean binary pattern like

and result in something like

11111110 11111110
01010110 00010101

Network Forensics

Have you been provided with a PCAP? Let’s start with

Introduction

Statistics > Capture File Properties provide the information about Capture Interface and Capture Filter and the hardware used for the capture.
Statistics > Protocol Hierarchy provides the different protocol captured. We can also use tshark -qz io,phs -r pcapfile.pcap
Statistics > Conversations/Endpoints provides good insight about the number of devices communicating.
Wireshark can extract objects transferred using FTP/HTTP/TFTP use File > Export Objects >. - The files can be extracted using --export-objects <protocol>,<destdir>. - For example: tshark -r filename.pcapng --export-objects "tftp,exported_objects" -2 > /dev/null
Wireshark can follow conversations using Analyse > TCP/HTTP/ Stream.
- We can use -z follow,prot,mode,filter[,range] where
  - prot specifies the transport protocol (tcp, udp, dccp, tls, http, http2, quic)
  - mode specifies the output mode (ascii, ebcdic, hex, raw, yaml)
  - filter specifies the stream to be displayed. There are three formats: (ip-addr0:port0,ip-addr1:port1, stream-index, stream-index,substream-index).
  - For example: tshark -r shark2.pcapng -qz follow,tcp,ascii,5
Wireshark can use display filter and extract field. - -Y|--display-filter <displaY filter> - -T ek|fields|json|jsonraw|pdml|ps|psml|tabs|text: Set the format of the output when viewing decoded packet data. - -e <field>: Add a field to the list of fields to display if -T ek|fields|json|pdml is selected. - For example: tshark -nr shark2.pcapng -Y 'frame contains "pico"' -T fields -e text
Fields can found using a particular protocol. For example Display Filter Reference: Domain Name System
We can save the output of a conversation (Follow TCP/UDP/HTTP/ and other streams), > Show data as and save as.

Wired PCAP

DNS Tunneling?

Check if there’s a DNS traffic to a public IP address. - For filtering only DNS queries: dns.flags.response == 0 - For filtering only DNS responses: dns.flags.response == 1

tshark -nr shark2.pcapng -Y 'dns && dns.flags.response == 0 && frame contains "local" && ip.dst==18.217.1.57' -T fields -e dns.qry.name

Decrypting traffic

TLS

Wireshark

Wireshark > Edit > Preferences > Protocols > TLS > RSA Key List and edit the private RSA key.

ssldump

ssldump - dump SSL traffic on a network

- r file   : Read data from file instead of from the network
-k keyfile : Use keyfile as the location of the SSL keyfile (OpenSSL format)
-d         : Display the application data traffic. This usually means decrypting it, but when -d is used ssldump will also decode application data traffic before the SSL session initiates.

Network attack Detection

Network attacks PCAP can have different kinds of scenarios - various host discovery techniques, - network port scanning methods, - various network attacks such as denial of service, poisoning, flooding and also - wireless attacks

Below information has mainly summarized from Detecting Network Attacks with wireshark.

Detection of host discovery (recon)

Technique	Wireshark Filter	Command / Tool
ARP scanning IP protocol scan ICMP ping sweep TCP ping sweeps UDP ping sweeps	arp.dst.hw_mac==00:00:00:00:00:00 icmp.type==3 and icmp.code==2 icmp.type==8 or icmp.type==0 tcp.dstport==7 udp.dstport==7	arp-scan -l nmap -sO <target> nmap -sn -PE <subnet> nmap -sn -PS/-PA <subnet> nmap -sn -PU <subnet>

Detection of network port scanning

Technique	Wireshark Filter	Command / Tool
TCP SYN scan TCP Connect() scan TCP Null scan TCP FIN scan TCP Xmass scan UDP port scan	tcp.flags.syn==1 and tcp.flags.ack==0 and tcp.window_size<=1024 tcp.flags.syn==1 and tcp.flags.ack==0 and tcp.window_size>1024 tcp.flags==0 tcp.flags==0x001 tcp.flags.fin==1 && tcp.flags.push==1 && tcp.flags.urg==1 icmp.type==3 and icmp.code==3	nmap -sS <target> nmap -sT <target> nmap -sN <target> nmap -sF <target> nmap -sX <target> nmap -sU <target>

To understand what ports were open on the attacked machine, we can use tcp.flags.ack==1 && tcp.flags.syn==1 && ip.src==ip_address_of_attacked_machine as after SYN packet machine sends SYN, ACK.

Detection of network attacks

Technique	Wireshark Filter	Command / Tool
ARP poisoning ICMP flood VLAN hoping Unexplained packet loss TCP Xmass scan UDP port scan	arp.duplicate-address-detected or arp.duplicate-address-frame icmp and data.len > 48 dtp or vlan.too_many_tags tcp.analysis.lost_segment or tcp.analysis.retransmission tcp.flags.fin==1 && tcp.flags.push==1 && tcp.flags.urg==1 icmp.type==3 and icmp.code==3	arpspoof, ettercap fping, hping frogger, yersinia n/a nmap -sX <target> nmap -sU <target>

Detection of wireless network attacks

Technique	Wireshark Filter	Command / Tool
Client deauthentication Client disassociation Fake AP beacon flood Authentication DoS UDP port scan	wlan.fc.type_subtype == 12 wlan.fc.type_subtype == 10 wlan.fc.type_subtype == 8 wlan.fc.type_subtype == 11 icmp.type==3 and icmp.code==3	aireplay-ng, mdk3, mdk4 mdk3, mdk4 mdk3, mdk4 mdk3, mdk4 nmap -sU <target>

Detection of reverse shell port

Few forensics challenges can request to find the reverse shell port number after the vulnerability was exploited. In that case, remember few things

If available, use Suricata logs to identify the time period when the attack happened.
If possible, identify what vulnerability was exploited (e.g. Eternal Blue exploits SMB) and search google for pcap analysis of that vuln to understand more.
Analyse the traffic after the vulnerability has been exploited.

EternalBlue

Eternalblue exploits SMB. So, we have to analyse smb traffic. - Look for smb.mid == 65 for EternalBlue and smb.mid == 81 or smb.mid == 82 for DoublePulsar? (This is unconfirmed).

TFTP Filter

tftp.opcode==1 : tftp Read Request

Wireshark tips

Many times, we are investigating a incident and knowing the exact time of the incident in wireshark is important. Use Menu > View > Time Display Format > Date and Time of the Day.

Wireshark - Searching for answers in pcap file?
Searching passwords in HTTP Web traffic in wireshark?

http.request.method == "POST" filter might help, based on concept that server is asking for LOGIN prompt and user is POSTing his password in cleartext.

Filters can be chained together using ‘&&’ notation. In order to filter by IP, ensure a double equals ‘==’ is used.
If the challenge says IP address has been spoofed, then you should look for MAC address as it wouldn’t have changed. You would find packets with two different IP address having same MAC address. In another scenario, if the MAC address has been spoofed, IP address might be the same. In both cases display filter “arp” (to only show arp requests) and “ip.addr==” (to show only packets with either source or destination being the IP address). might be helpful.
Sometimes, it is better to check which objects we are able to export, (File –> Export Objects –> HTTP/DICOM/SMB/SMB2) export the http/DICOM/SMB/SMB2 object
SSL Traffic? and have a key? Visit Wireshark->Edit->Preferences->Protocols->SSL->RSA Key List. SSL Traffic with forward secretcy ->SSL->Pre-Master-Secret-Log filename
Sometimes, you need to find all the unique ip address in the network capture, for that you can use

 tshark -T fields -e ip.src -r <pcap file> \| sort \| uniq

-T fields\|pdml\|ps\|psml\|text : Set the format of the output when viewing decoded packet data.
-e : Add a field to the list of fields to display if -T fields is selected.
-r : Read packet data from infile, can be any supported capture file format (including gzipped files).
-R : Cause the specified filter (which uses the syntax of read/displayfilters, rather than that of capture filters) to be applied

Wireshark can not reassamble HTTP fragmented packets to generate the RAW data,we can use Dshell to reassemble http partial contents. A blog mentioning how to do it is here
If there’s any file getting transferred in the PCAP, maybe try carving out using binwalk or foremost, you might get lucky.

If you are searching for flags, and nothing is quite obvious.
- Check for hints such a packet mentioning start or end.
- Check for patterns in source port number (sport changing with same destination port) or destination port number (dport changing with same source port).

Finding information

Bittorrent

If we have bittorrent traffic and want to find the file being downloaded, look for BT-DHT packets and look for info_hash. Search the hash on internet.

Wireless PCAP

The assumption is that the wireless PCAP would be encrypted. In this section, we will see the encryption

aircrack-ng can crack WEP/WPA handshakes using a word dictionary specified using -w.

aircrack-ng wpa-ing_out.pcap -w /usr/share/wordlists/rockyou.txt

Once we have found the key (password), go to Wireshark > Edit > Preferences > Protocols > IEEE 802.11 and edit the decryption key.

USB Forensics

Probably, we would be provided with the USB-based PCAP file, now as there are USB-Mouse/ Keyboard and Storage devices. There would be data related to that. Now, to figure what device is connected. Check the below packets in the wireshark

1      0.000000        host    1.12.0  USB     36      GET DESCRIPTOR Request DEVICE
2      0.000306        1.12.0  host    USB     46      GET DESCRIPTOR Response DEVICE

In the GET DESCRIPTOR Response packet, there would be a idVendor and idProduct, searching for that. We can figure out that whether it’s a Keyboard, mouse or storage device.

DEVICE DESCRIPTOR
   bLength: 18
   bDescriptorType: 0x01 (DEVICE)
   bcdUSB: 0x0200
   bDeviceClass: Device (0x00)
   bDeviceSubClass: 0
   bDeviceProtocol: 0 (Use class code info from Interface Descriptors)
   bMaxPacketSize0: 8
   idVendor: Razer USA, Ltd (0x1532)
   idProduct: BlackWidow Ultimate 2013 (0x011a)
   bcdDevice: 0x0200
   iManufacturer: 1
   iProduct: 2
   iSerialNumber: 0
   bNumConfigurations: 1

USB-Keyboard

If the device connected is the keyboard, we can actually, check for the “interrupt in” message

51     8.808610        1.12.1  host    USB     35      URB_INTERRUPT in

and check for the Leftover Capture Data field

Frame 159: 35 bytes on wire (280 bits), 35 bytes captured (280 bits)
USB URB
   [Source: 1.12.1]
   [Destination: host]
   USBPcap pseudoheader length: 27
   IRP ID: 0xffffa5045d1653c0
   IRP USBD_STATUS: USBD_STATUS_SUCCESS (0x00000000)
   URB Function: URB_FUNCTION_BULK_OR_INTERRUPT_TRANSFER (0x0009)
   IRP information: 0x01, Direction: PDO -> FDO
   URB bus id: 1
   Device address: 12
   Endpoint: 0x81, Direction: IN
   URB transfer type: URB_INTERRUPT (0x01)
   Packet Data Length: 8
   [bInterfaceClass: HID (0x03)]
Leftover Capture Data: 0000500000000000

Now, we can use tshark to take out, usb.capdata out

tshark -r usb-keyboard-data.pcap -T fields -e usb.capdata
00:08:00:00:00:00:00
00:00:00:00:00:00:00
00:0e:00:00:00:00:00
00:00:00:00:00:00:00
00:16:00:00:00:00:00

Here there are 8 bytes

Keyboard Report Format

Byte 0: Keyboard modifier bits (SHIFT, ALT, CTRL etc)
Byte 1: reserved
Byte 2-7: Up to six keyboard usage indexes representing the keys that are currently “pressed”. Order is not important, a key is either pressed (present in the buffer) or not pressed.

USB HID Keyboard Scan Codes

MightyPork has created a gist mentioning USB HID Keyboard scan codes as per USB spec 1.11 at usb_hid_keys.h

The above can be referred and utilized to convert the usb.capdata to know what was the user typing using the USB Keyboard!

whoami has written a script to figure out the keyboard strokes

usb_codes = {
   0x04:"aA", 0x05:"bB", 0x06:"cC", 0x07:"dD", 0x08:"eE", 0x09:"fF",
   0x0A:"gG", 0x0B:"hH", 0x0C:"iI", 0x0D:"jJ", 0x0E:"kK", 0x0F:"lL",
   0x10:"mM", 0x11:"nN", 0x12:"oO", 0x13:"pP", 0x14:"qQ", 0x15:"rR",
   0x16:"sS", 0x17:"tT", 0x18:"uU", 0x19:"vV", 0x1A:"wW", 0x1B:"xX",
   0x1C:"yY", 0x1D:"zZ", 0x1E:"1!", 0x1F:"2@", 0x20:"3#", 0x21:"4$",
   0x22:"5%", 0x23:"6^", 0x24:"7&", 0x25:"8*", 0x26:"9(", 0x27:"0)",
   0x2C:"  ", 0x2D:"-_", 0x2E:"=+", 0x2F:"[{", 0x30:"]}",  0x32:"#~",
   0x33:";:", 0x34:"'\"",  0x36:",<",  0x37:".>", 0x4f:">", 0x50:"<"
   }
lines = ["","","","",""]

pos = 0
for x in open("data1.txt","r").readlines():
   code = int(x[6:8],16)

   if code == 0:
       continue
   # newline or down arrow - move down
   if code == 0x51 or code == 0x28:
       pos += 1
       continue
   # up arrow - move up
   if code == 0x52:
       pos -= 1
       continue
   # select the character based on the Shift key
   if int(x[0:2],16) == 2:
       lines[pos] += usb_codes[code][1]
   else:
       lines[pos] += usb_codes[code][0]


for x in lines:
   print x

USB-Mouse

If we take the USB-Mouse Leftover Capture data, we have around four bytes

Format of First 3 Packet Bytes

Even if your mouse is sending 4 byte packets, the first 3 bytes always have the same format. * The first byte has a bunch of bit flags.

byte 1:
Y overflow    X overflow      Y sign bit      X sign bit      Always 1        Middle Btn      Right Btn       Left Btn

The second byte is the “delta X” value – that is, it measures horizontal mouse movement, with left being negative.

byte 2:
X movement

The third byte is “delta Y”, with down (toward the user) being negative. Typical values for deltaX and deltaY are one or two for slow movement, and perhaps 20 for very fast movement. Maximum possible values are +255 to -256 (they are 9-bit quantities, two’s complement).

byte 3:
Y movement

Let’s say we capture this data into a file, we can eventually capture the mouse movements,

tshark -r challenge.pcapng usb.capdata and usb.device_address==12 -T fields -e usb.capdata > mouse_data.txt

This can be plotted using GNUplot as shown in a writeup of Riverside

awk -F: 'function comp(v){if(v>127)v-=256;return v}{x+=comp(strtonum("0x"$2));y+=comp(strtonum("0x"$3))}$1=="01"{print x,y}' mouse_data.txt > click_coordinates.txt

GNUplot

gnuplot -e "plot 'click_coordinates.txt'"

If the mouse movement shows a on-screen keyboard, probably, we can use

awk 'BEGIN{split("          zxcvbnm  asdfghjkl qwertyuiop",key,//)}{r=int(($2-20)/-100);c=int(($1 - 117 + (r % 2 * 40)) / 85);k=r*10+c;printf "%s",key[k]}END{print""}' click_coordinates.txt

USB-Storage-Device

If the device found in the PCAP is a USB-Storage-Device, check for the packets having size greater than 1000 bytes with flags URB_BULK out/in. Select the stream and press Ctrl + h or you can use File->Export Packet Bytes.

Browser Forensics

Browser extension

ExtAnalysis : Browser Extension Analysis Framework - Scan, Analyze Chrome, firefox and Brave extensions for vulnerabilities and intels.

File Forensics

Webserver logs

GoAccess : GoAccess is an open source real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems or through your browser.

ZIP Files

Shar creates “shell archives” (or shar files) which are in text format and can be mailed. These files may be unpacked later by executing them with /bin/sh.

Word/Powerpoint and others? Macro

Word/Powerpoint files can be opened using zip software like 7z x hello.doc.
Further it is always a good idea to list all the extracted files using ls -laR.
After that, we can use ViperMonkey which is a VBA Emulation engine written in Python, designed to analyze and deobfuscate malicious VBA Macros contained in Microsoft Office files (Word, Excel, PowerPoint, Publisher, etc).

PDF Files

Redacted PDF

Masking over the text of a PDF document does not actually erase the underlying text.

Use pdftotext to extract the text from pdf and see if there’s a flag.

Memory Forensics

Volatility

Command Reference

Important commands to try

imageinfo/ pslist / cmdscan/ consoles/ consoles/ memdump/ procdump/ filescan/ connscan/
Extract files using filescan and dumpfiles

https://security.stackexchange.com/questions/256283/how-to-identify-hidden-processes-with-volatility-using-psxview

Extracting RAW pictures from memory dumps

Extracting RAW pictures from Memory Dumps

Probably, dump the process running MSRDP, MSPAINT

Rename the file extensions from *.dmp to *.data, download/install GIMP and open them as “RAW Image Data”:
We can use GIMP to navigate within the memory dump and analyse the rendered pixels/bitmaps on their corresponding offsets

Disk Forensics

Initial recon

If we are provided a image,

img_stat to display details of an image file.

└─$ img_stat dds2-alpine.flag.img
IMAGE FILE INFORMATION
--------------------------------------------
Image Type: raw

Size in bytes: 134217728
Sector size:    512

mmls to display the partition layout of a volume system (partition tables). If mmls doesn’t provide any output, run fsstat on the disk image as it might not have any partition table.

mmls dds2-alpine.flag.img
DOS Partition Table
Offset Sector: 0
Units are in 512-byte sectors

      Slot      Start        End          Length       Description
000:  Meta      0000000000   0000000000   0000000001   Primary Table (#0)
001:  -------   0000000000   0000002047   0000002048   Unallocated
002:  000:000   0000002048   0000262143   0000260096   Linux (0x83)

fsstat to display general details of a file system. The output of this command is file system specific. At a minimum, the range of meta-data values (inode numbers) and content units (blocks or clusters) are given. Also given are details from the Super Block, such as mount times and and features. It requires -o imgoffset (the sector offset where the file system starts in the image).

fsstat -o 2048 dds2-alpine.flag.img
FILE SYSTEM INFORMATION
--------------------------------------------
File System Type: Ext3
Volume Name:
Volume ID: dc53a3bb0ae739a5164c89db56bbb12f

Last Written at: 2021-02-16 13:21:20 (EST)
Last Checked at: 2021-02-16 13:21:19 (EST)

Last Mounted at: 2021-02-16 13:21:19 (EST)
Unmounted properly
Last mounted on: /os/mnt

Source OS: Linux
Dynamic Structure
Compat Features: Journal, Ext Attributes, Resize Inode, Dir Index
InCompat Features: Filetype,
Read Only Compat Features: Sparse Super, Large File,

Journal ID: 00
Journal Inode: 8

fls to list file and directory names in a disk image.

usage: fls [-adDFlhpruvV] [-f fstype] [-i imgtype] [-b dev_sector_size] [-m dir/] [-o imgoffset] [-z ZONE] [-s seconds] image [images] [inode]
        If [inode] is not given, the root directory is used
        -a: Display "." and ".." entries
        -d: Display deleted entries only
        -D: Display only directories
        -F: Display only files
        -o imgoffset: Offset into image file (in sectors)
        -r: Recurse on directory entries
        -u: Display undeleted entries only
        -k password: Decryption password for encrypted volumes

For example:

fls -a -o 2048 dds2-alpine.flag.img
d/d 26417:      home
d/d 18290:      root
d/d 16259:      run

Then fls can be used see files in particular Folders by providing the above inode number.

fls -a -o 2048 dds2-alpine.flag.img 18290
d/d 18290:      .
d/d 2:  ..
r/r 18291:      down-at-the-bottom.txt

Use icat to read file or output the contents of a file based on its inode number. From above we get the inode number of the file.

icat -o 2048 dds2-alpine.flag.img 18291
  / \   / \   / \   / \   / \   / \   / \   / \   / \   / \   / \   / \   / \
 ( p ) ( i ) ( c ) ( o ) ( C ) ( T ) ( F ) ( { ) ( f ) ( 0 ) ( r ) ( 3 ) ( n )
  \_/   \_/   \_/   \_/   \_/   \_/   \_/   \_/   \_/   \_/   \_/   \_/   \_/

Further, there could be slack space in the file. We can use -s to include the slack space.

icat -s suspicious.dd.sda1 12
Nothing to see here! But you may want to look here -->
}1937befc_3<_|Lm_111t5_3b{FTCocip

Mounting the disk

Method 1

Install image as loops and mount the partition required.

[user@host Operation Oni]$ sudo kpartx -a -v disk.img
add map loop19p1 (253:22): 0 204800 linear 7:19 2048
add map loop19p2 (253:23): 0 264192 linear 7:19 206848
[user@host Operation Oni]$ sudo mount /dev/mapper/loop19p2 mnt/

We can browse the mounted directory using cd.

Unmount the disk and delete the image as loop.

[user@host Operation Oni]$ umount mnt
[user@host Operation Oni]$ sudo kpartx -d disk.img
loop deleted : /dev/loop19

Method 2

disk -l disk.img
Disk disk.img: 230 MiB, 241172480 bytes, 471040 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x0b0051d0

Device     Boot  Start    End Sectors  Size Id Type
disk.img1  *      2048 206847  204800  100M 83 Linux
disk.img2       206848 471039  264192  129M 83 Linux

Mount the disk by replacing the offset. In the below, we just replace the Start number to calculate the offset.

sudo  mount -o loop,offset=`echo 206848 \* 512|bc` disk.img /mnt

RAID

Redundant Array of Inexpensive Disks

RAID can be used for a number of reasons such as squeezing out extra performance, offering redundancy to your data and even parity; parity is what rebuilds data which is potentially lost, thus offering an extra level of protection from data loss.

The most common types of RAID array are

RAID 0

Requires a minimum of 2 disks to create

Widely known as the performance RAID

Offers no redundancy whatsoever (no mirroring or parity featured)

RAID 1

Like RAID 0, requires a minimum of 2 disks to create

Offers good redundancy due to RAID 1 using a mirrored drive

RAID 5

Requires a minimum of 3 disks to setup

Gives a level added of redundancy through parity

RAID 10 (Sometimes known as RAID 1+0)

A minimum of 4 disks is needed

Effectively RAID10 is a RAID0 and 1 array combined into a single arra

Challenges

If we are provided either two or three raid disk file in which one is crashed, we can eventually recover it.

$file disk*
disk0:    DOS/MBR boot sector, code offset 0x3c+2, OEM-ID "mkfs.fat", sectors/cluster 4, root entries 512, sectors 2048 (volumes &lt;=32 MB) , Media descriptor 0xf8, sectors/FAT 2, sectors/track 32, heads 64, reserved 0x1, serial number 0x867314a9, unlabeled, FAT (12 bit)
disk1:    ASCII text
disk2:    data

$ ls -lh
512K  disk0
12    disk1
512K  disk2

$ cat disk1
crashed :-()

From above output we know that disk1 is missing. We also know that RAID was used. The most probable version of RAID allowing 1 out of 3 disk loss is the one where every disk can be obtained by XOR-ing 2 other disks. We XOR-ed disk0 and disk2 to get disk1 using some python:

from pwn import *
with open("disk0", "rb") as f1:
   with open("disk2", "rb") as f2:
       with open("disk1", "wb") as f3:
           x = f1.read()
           y = f2.read()
           f3.write(xor(x,y))

or we can use xor-files to XOR for two or more files and get the result on a pipe

Now, to get the full NAS content, we had to determine the block distribution. After few minutes of analyzing the disks content and with some knowledge of FAT12 structure) we have determined that parity block (BP) is on different disk in each row so we have distribution:

D0 | D1 | D2
---|----|---
B0 | B1 | BP
B2 | BP | B3
BP | B4 | B5
B6 | B7 | BP

Simple python code to piece together all data blocks:

n = 1024
k = 512    # block size

with open("disk0", "rb") as f1:
   with open("disk1", "rb") as f2:
       with open("disk2", "rb") as f3:
           with open("disk_out", "wb") as f_out:
               x = 2
               for _ in xrange(n):
                   blocks = (f1.read(k), f2.read(k), f3.read(k))
                   data_blocks = [b for i, b in enumerate(blocks) if i != x]
                   x = (x - 1) % 3
                   f_out.write("".join(data_blocks))

Now to check the content we can mount the resulting disk image:

$ sudo mount disk_out  /mnt/img/

Formats

Boarding Pass Format

Boarding pass issued at the airport from What’s contained in a boarding pass barcode?

M1EWING/SHAUN         E1AAAAA SYDBNEQF 0524 106Y023A0073 359>2180
B                29             0    QF 1245678             128

There’s more information in this boarding pass barcode, which is as follows:

M1 : Format code ‘M’ and 1 leg on the boarding pass.
EWING/SHAUN : My name.
E1AAAAA : Electronic ticket indicator and my booking reference.
SYDBNEQF : Flying from SYD (Sydney) to BNE (Brisbane) on QF (Qantas).
0524 : Flight number 524.
106 : The Julian date. In this case 106 is April 16.
Y : Cabin – Economy in this case. Others including F (First) and J (Business).
23A : My seat.
0073 : My sequence number. In this case I was the 73rd person to check-in.
3 : My “passenger status”.
59 : There is a various size field. This is the size
> : Beginning of the version number
2 : The version number.
18 : Field size of another variable field.
0 : My check-in source.
B : Airline designator of boarding pass issuer.
2 : Another variable size field.
9 : Airline code.
0 : International document verification. ’0′ as I presume is not applicable.
QF : The airline my frequent flyer account is with.
1245678 : My frequent flyer number.
128 : Airline specific data.

Interesting Blog

Others

The Konami Code is a cheat code that appears in many Konami video games, although the code also appears in some non-Konami games. The player could press the following sequence of buttons on the game controller to enable a cheat or other effects:

[38, 38, 40, 40, 37, 39, 37, 39, 66, 65, 66, 13] is actually: UP UP DOWN DOWN LEFT RIGHT LEFT RIGHT B A ENTER

A000045 would bring up the fibonacci numbers.
Unicode
In a TCP Dump, you see a telnet session entering login username and password and those creds are not valid. Maybe check the value in HEX. If it contains 0x7F, that’s backspace.
If in a challenge, you are provided a setgid program which is able to read a certain extension files and flag is present in some other extension, create a symbolic link to the flag with the extension which can be read by the program. For example: In picoCTF 2014 Supercow challenge, a program named supercow was able to read files with .cow extension only and flag was present with flag.txt. So we created a symbolic link like ln -s flag.txt flag.cow
If in a challenge, you are provided with a APK file. There are three ways to decompile it as described below:

Apktool: It is used to decode resources to nearly original form (including resources.arsc, XMLs and 9.png files) and rebuilding them. Also, used for smali debugging. apktool converts the apk file in to smali format. smali/baksmali is an assembler/disassembler for the dex format used by dalvik, Android’s Java VM implementation.
apktool d file.apk output-dir
d : decode to output-dir
Dex2jar: To see the java code (approx)

Change the extension of file.apk from .apk to .zip

Unzip the file.zip

After unzip, you would get classes.dex file.

Use dex2jar classes.dex (It would create classes_dex2jar.jar file)

Extract jar file by jar xf classes_dex2jar.jar

This would provide you with .class files which could be open by jd-gui (Java Decompiler) tool.

Use online services such as Decompile Android. Once it’s decompiled, we can download the decompiled files and unpack them.

If you are provided a disk.img file, from which files have to recovered, you could use foremost tool used to recover files using their headers, footers, and data structures.

If you are provided with iOS package, we may use dpkg-deb to extract it.

dpkg-deb -x com.yourcompany.whyos_4.2.0-28debug_iphoneos-arm.deb app

If you are provided a jar file in the challenge, JAR (Java ARchive) is a package file format typically used to aggregate many Java class files and associated metadata and resources (text, images, etc.) into one file to distribute application software or libraries on the Java platform. It can be extracted using

jar xf jar-file
x : extract files from the JAR archive.
f : JAR file from which files are to be extracted is specified on the command line, rather than through stdin.
The jar-file argument is the filename (or path and filename) of the JAR file from which to extract files.

If you are having a source code of evil program, check the source code of the real program, do a comparision and find the added evil code.
Morse code, utilize Transator
Sometimes, if you extract some files, if you wuld see a blank name, you know there is some file but can’t see a name, like file name could be spaces?, then

ls -lb might be of help.
-b, --escape :   print C-style escapes for nongraphic characters

How to open a filename named “-” : We can create a file named “-” by

echo hello > -
and this file can be opened by
cat ./-
This is needed because lot of programs use “-” to mean stdin/stdout.

If you have a hex dump of something and you want to create the binary version of the data?

xxd r data
data is the hexdump of the binary file.

Excel Document: You may try unzipping it and check VBA macros in it. There are tools to extract VBA from excel listed here ools to extract VBA Macro source code from MS Office Documents
GIF to JPG

convert animation.gif target.jpg

If the pdf-parser contains

/ProcSet [/PDF/Text/ImageC/ImageI]
/ProcSet [/PDF/Text/ImageC/ImageI]

It means it will contain text which can be extracted by using

*pdf2txt Untitled-1_1a110935ec70b63ad09fec68c89dfacb.pdf
 PCTF{how_2_pdf_yo}*

If you get an IP address on the challenge and probably no port is open and pinging, try to check the response time of the pings, it might different each time and maybe representing binary 0 (If response time is less than Xms) or binary 1 (If the response time is greater than Xms).

Forensics Tools used

AutoRuns PowerShell Module <https://github.com/p0w3rsh3ll/AutoRuns>: AutoRuns module was designed to help do live incident response and enumerate autoruns artifacts that may be used by legitimate programs as well as malware to achieve persistence.

Sample Entry:

<item>
<location>HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders\Common Startup</location>
<itemname>Windows Updater.lnk</itemname>
<enabled>Enabled</enabled>
<profile>System-wide</profile>
<launchstring>C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Startup\Windows Updater.lnk</launchstring>
<imagepath>c:\users\barry\downloads\updater.cmd</imagepath>
<time>20230504-124939</time>
</item>

Firmware image provided? with Squashfs? Use unsquashfs to uncompress, extract and list squashfs filesystems

CONTROL: BOA archive data
kernel:  Device Tree Blob version 17, size=2640268, boot CPU=0, string block size=108, DT structure block size=2639208
root:    Squashfs filesystem, little endian, version 4.0, xz compressed, 11550948 bytes, 1199 inodes, blocksize: 262144 bytes, created: Tue Jan 31 15:36:57 2023