aacRepair

repairs aac and aacPlus files grabbed from the internet

Contents

aacRepair - Full documentation

repair aac and aacPlus files grabbed from the internet


Info

AAC files consist of multiple segments, frames. Each frame has a header and a payload. Browser gets stuck if AAC file frame is defective and will not start to play or refuse to play next AAC file. This will stop the entire playlist. File gets trimmed from head to tail, to remove defective frames. Cut off byte count is shown in the summary.

Note

Self documenting code is (docStrings) short and read by document builder tools. –> aacrepair package

Command Line

$ aacrepair

        Menu "Main"
1 -- Single File aac or aacPlus
2 -- Bulk Repair, Folder
3 -- Exit
Enter your choice: 1

Python or sys.path fail; use

$ python -m aacrepair.cmd

aacrepair Module

bulk repair

from aacrepair import AacRepair

# 'r' before a string tells the Python interpreter to treat backslashes as a literal (raw) character
aacRepair = AacRepair(r"F:\propaganda-podcasts")
# setter overrides default export path 'F:\propaganda-podcasts\aac_repair'
aacRepair.set_export_path(r"F:\repaired_foobar")
aacRepair.repair()

Instantiate AacRepair class with two possible arguments, mandatory folder path and optional dictionary.

1. No dictionary provided. Folder path is used as list to import files into a dictionary AND store repaired files.

2. A dictionary of files is provided. Folder path is used to store repaired files. (best use on web server)

Web Server

• Endpoint converts multi-file upload from file storage type to bytestream, uses .read() method

• List of files is written to dictionary {file_name_key: file_byte_content_value}

• web server gets not the file path, only file name - needs path to store repaired files

• dictionary {file(n).aac: b’x65x66x67x00x10x00x00x00x04x00’}

code:

files = request.files.getlist('fileUploadAcpRepair')
f_dict = {f: open(f, "rb").read() for f in files if f[-5:] == ".aacp" or f[-4:] == ".aac"}
aacRepair = AacRepair("/home/Kitty/aac_files", f_dict)
aacRepair.repair()

File System

• List of files in folder is written to dictionary {file_name_key: file_byte_content_value}

code:

aacRepair = AacRepair("/home/Kitty/meow_aac")
aacRepair.set_export_path("/home/Kitty/foo")
aacRepair.repair()

single object

head and tail are used to cut chunks left or right only

aacRepair = AacRepair()
# converts file path to file content, if object is not of type bytes
rep_object = aacRepair.repair_object(aac_path_or_object, head=None, tail=None)

header_aac Module

Use as input for further aac stream processing or repair.

from aacrepair import header_aac

header_aac module example to show all frames with header.

header_aac.pull_frame('/home/foo/bar.aac')

header_aac dictionary output of header_info(aac_object, frame_bytes=None, print_out=True)

SYNC_WORD_BOOL: True
MPEG4_BOOL: True
Layer_BOOL: True
CRC_16_IS_SET_BOOL: False
PROFILE_INT: 1
PROFILE_STR: AAC Main
SAMPLING_FREQUENCY_INT: 3
SAMPLING_FREQUENCY_STR: 48000 Hz
PRIVATE_BIT_BOOL: False
CHANNEL_CONFIG_INT: 2
CHANNEL_CONFIG_STR: 2 channels: front-left, front-right
ORIGINALITY_BOOL: False
HOME_BOOL: False
COPYRIGHT_ID_INT: 0
COPYRIGHT_START_INT: 0
FRAME_LENGTH_INT: 530
BIT_RESERVOIR_INT: 2047
FRAME_NUMBER_INT: 0
CRC_16: {}
IS_LAST_FRAME_BOOL: False
ERROR_STR:
FRAME_BYTES: b''

Bytes

1	2	3	4	5	6	7	8	9
AAAAAAAA	AAAABCCD	EEFFFFGH	HHIJKLMM	MMMMMMMM	MMMOOOOO	OOOOOOPP	QQQQQQQQ	QQQQQQQQ

Bit Groups

Group	Number	Count	Description
A	0-12	12	Syncword, all bits 1
B	13	1	MPEG Version set 0 is MPEG-4, set 1 MPEG-2
C	14-15	2	Layer set to 0
D	16	1	[[[ Warning ]]], set to 1 if there is no CRC and 0 if there is CRC
E	17-18	2	Profile, the MPEG-4 Audio Object Type https://en.wikipedia.org/wiki/MPEG-4_Part_3
F	19-22	4	MPEG-4 Sampling Frequency https://wiki.multimedia.cx/index.php/MPEG-4_Audio
G	23	1	Private bit set to 0,
H	24-26	3	MPEG-4 Channel Configuration https://wiki.multimedia.cx/index.php/MPEG-4_Audio
I	27	1	Originality, set 1 originality of audio, else 0
J	28	1	Home, set to 1 to signal home usage of the audio, else 0
K	29	1	Copyright ID bit
L	30	1	Copyright ID start
M	31-43	13	Frame length of ADTS frame including headers and CRC check - 1
O	44-54	11	Buffer fullness, states the bit-reservoir per frame
P	55-56	2	Number of AAC ADTS frame minus 1
Q	57-72	16	CRC check

pip install

""" Linux """
$ pip3 install aacrepair

""" Windows """
> pip install aacrepair

Uninstall

Python user

• find the module location

• uninstall and then remove remnants

remove:

>$ pip3 show aacrepair
>$ pip3 uninstall aacrepair

Location: … /python310/site-packages

What’s next - contributions welcome

• try multithread a test

• multiprocessing plus multithreading repair, if file count is x (works only on linux)

• android ‘Threading in Worker’, make it run

aacrepair

aacrepair package

Submodules

aacrepair.audio_conf module

Translate Bit representation of audio properties into readable strings.

• “PROFILE_INT”: 3

• “PROFILE_STR”: AAC SSR (Scalable Sample Rate)

aacrepair.cmd module

Module runs aacrepair on commandline with menu options.

Please check error_dict message, if instance fails to repair.

aacrepair.cmd.bulk_repair()

Input loop to repair a whole folder of aac files.

Methods:

prepare_path_write_bulk: create instance with folder and export path argument

aacrepair.cmd.file_repair()

Input loop to repair a single aac file.

Methods:

repair_write_one_file: read, repair and store renamed file

aacrepair.cmd.instance_repair_bulk(aac_path, export_path=None)

Create instance with option for custom export folder.

Params:

aac_path: src directory

Params:

export_path: this function calls the setter to change export directory

aacrepair.cmd.main()

Call menu_main to start the module from command line.

aacrepair.cmd.menu_main()

Main menu to choose from.

aacrepair.cmd.prepare_path_run_write_bulk(aac_path)

Prepare path arguments to feed the repair instance

Params:

aac_path: src directory arg[0] dst directory arg[1]

Methods:

instance_repair_bulk: create instance with default folder or export path

aacrepair.cmd.repair_write_one_file(aac_path, name_prefix=None, print_out=True)

Repair a single aac or aacPlus file and writes it with a name prefix.

Param:

aac_path: file path

Param:

name_prefix: distinguish the file from defective one

Param:

print_out: None disables print to screen

Methods:

header_dict.header_info: header bits to property dictionary

Returns:

AAC frame header for further processing

Return type:

dict

aacrepair.crc module

Should help to read crc bytes and calculate checksum of a container and content.

Place to calculate checksum of Audio container and/or its content.

aacrepair.crc.flip_this(bit_str)

Complement 1’s (flip 0 to 1, 1 to 0) string of bits.

Param:

bit_str: left to right ordered string of all CRC fields

Returns:

1’s complement

Return type:

str

aacrepair.crc.reveal_crc(byte_list, print_out=None)

Helper to create a function for AAC stream CRC check. Calc all possible bit orders, to detect the CRC-CCITT input function pre return options used; Normal, reflected and/or reversed. Then use either Python CRC check, or a Pypi packet. aacrepair.header_aac module can return hex output to put it in an online CRC checker or use fastCRC packet. Output must be altered bytes array to slice in the (clean-up before calc) crc sum fields ‘ffff’, if CRC bit is set. Document ‘ETSI TS 102 563’ section 5.2, Transport of Advanced Audio Coding (AAC) audio; Technical Specification Perhaps we can create a software error correction ourselves. Document ‘header_firecode’, perhaps go from there.

Params:

byte_list: CRC SUM as list of 8 bits of type bytearray or bytes [8], [8,8], [8,8,8,8], [8,8,8,8,8,8,8,8]

Params:

print_out: disable screen printing

Returns:

CRC SUM as string, hex and int representation of left right, right left, reversed and flipped bites order

Return type:

dict

aacrepair.dev_aac module

aacrepair.haeder_mp4 module

aacrepair.header_aac module

Output is human-readable. Non-destructive. Use as input for further aac stream processing.

• check next fame is available

• check if profile changed to low quality stream

• compare sample rates to keep the better stream

• switch the stream url endpoint on low quality, if channel config changed

aacrepair.header_aac.header_index_get(aac_object)

Scan the file object for an aac frame. Search frame is hex.

Param:

aac_object: bytes

Returns:

INDEX number of the first frame start Byte in the stream

Return type:

int

aacrepair.header_aac.header_info(aac_object, frame_bytes=None, print_out=None)

Caller can name a slice of the object aac_object[start idx:] to work. Example how to move the header_info in header_aac.read_all_header().

Param:

aac_object: full object or slice

Param:

frame_bytes: dump whole frame to header_dict[‘FRAME_BYTES’]

Param:

print_out: enable print to screen

Returns:

frame header properties

Return type:

dict

aacrepair.header_aac.main()

aacrepair.header_aac.pull_frame(path_str=None)

Example to get the header and all frames of the object (file content).

Param:

path_str: file path or object

aacrepair.header_aac.read_all_header(aac_object, convert_bytes_hex=None)

Example function returns all frame content with header until empty.

Param:

aac_object: bytes type object

Param:

convert_bytes_hex: for cut-and-paste into checksum calculator

Returns:

header information and object content

Return type:

Iterator[dict]

Module contents

Module repairs aac or aacPlus file objects. Stores on disk or returns in memory.

technical:

AAC file wants a nice head and tail.
AAC header starts hex fff(1) or (9), mpeg-(4) or (2)
Create a header search frame binary b'ÿñ', hex fff1
Move the header search frame over the AAC file.
Search start is file[0:2] bytes, shift the search frame file[1:3], file[2:4]
AAC head: remove the first defective frame ...[fff1 file]
AAC tail: remove the last frame with defective payload [file]fff1...

class aacrepair.AacRepair(folder=None, file_dict=None)

Bases: object

Write repaired aac or aacPlus file objects to disk or memory.

__init__

Params:

folder: use folder for dict if file_dict is None, else is export_path

Params:

file_dict: {file name or path: content}, folder is export_path

Methods:

file_dict_from_folder: read content of aac files folder into a dict for bulk repair

Class Methods

Methods:

set_export_path: setter export_path

Methods:

get_export_path: getter export_path

Methods:

file_dict_from_folder: {file_path: content,}

Methods:

make_dirs: create a folder with subfolders

Methods:

repair: threaded call of repair_one_file with path, content args

Methods:

convert_file_like_object: return a bytes type object

Methods:

repair_object: single file or chunk, dispatcher

Methods:

repair_one_file: head and tail repair

Methods:

repair_head: only head

Methods:

repair_tail: only tail

Methods:

log_writer: log_list as result

Methods:

byte_calc: calc cut bytes

Methods:

write_repaired_file: write object to file

Methods:

delete_file_dict: del file_dict

Methods:

all_files_touched: if fail generate report

Methods:

report_skip_list: skipped files not in error_dict; shall reveal module failure

all_files_touched()

Check for error in module, file failure not written in error_dict.

Returns:

False if calc not match, see report_skip_list

byte_calc(f_name)

Return number of cut bytes.

Param:

f_name: file path for error, or info dict

Exception:

write error, set size to 1

Return type:

int

Returns:

cut bytes

Return type:

int

convert_file_like_object(file_full_name)

Read file from file system and return content as bytes type.

Param:

file_full_name: The absolute path to the file

Exception:

write report into error_dict

Return type:

False

Returns:

object type bytes

delete_file_dict()

Outsourced to prevent del dict in test mode.

file_dict_from_folder()

Create dictionary of files {name: content} for the ‘repair()’ method, bulk repair.

Class can also use an existing dictionary (prepared by web server multi-file upload)

get_export_path()

Getter for export_path dir.

log_writer()

Write available logs to screen and keep it log_list for later HTML colorized report.

Return type:

True: ok

Return type:

False: errors found, or files where skipped due to a program bug

static make_dirs(path)

Create folders.

Param:

path: absolute path that can contain subdirectories

repair(keep_file_dict=None)

Threaded bulk repair from dictionary.

Params:

keep_file_dict: delete the dictionary yourself

Return type:

True; if keep_file_dict manual delete_file_dict

Return type:

False; check skip_list and error_dict, then –> delete both

repair_head(f_name, chunk)

Return bytes content left repaired …[fff1 file]

Param:

f_name: file name for the error, info dict

Param:

chunk: The data to cut.

Returns:

binary data

Return type:

bytes

repair_object(chunk, head=None, tail=None)

Return the repaired binary file object from binary source (buffer) or file content.

file_full_name is alias to reuse methods and write to error and info dicts

Params:

chunk: buffer, queue, or file content

Params:

head: of chunk repair

Params:

tail: of chunk repair

Methods:

convert_file_like_object: get file content from path given

Methods:

repair_one_file: one chunk left and right, call repair_head and repair_tail

Methods:

repair_head: left side only

Methods:

repair_tail: right side only

Return type:

bytes type file object

Return type:

False; check error_dict after this method call

repair_one_file(file_full_name, damaged_data, skip_write=None)

Repair the beginning or end of file (damaged_data is dictionary value).

Params:

file_full_name: The absolute path to the file.

Params:

damaged_data: binary file content

Params:

file_full_name: bulk repair, name of export folder

Params:

skip_write: method should not write a file

Return type:

bytes: binary data: repaired file content

Return type:

False: read the error dict

repair_tail(f_name, chunk)

Return bytes content right repaired [file]fff1…

Param:

f_name: file name for the error, info dict

Param:

chunk: The data to cut.

Returns:

binary data

Return type:

bytes

report_skip_list()

all_files_touched sum calc got no match, write file names of skipped files into skip_list

Returns:

list of skipped files

Return type:

list

set_export_path(path)

Setter for export_path dir attribute.

static write_repaired_file(export_path, file_content)

Write repaired file content to disk.

Param:

export_path: setter or os.joined default export path

Param:

file_content: bytes type content

Indices and tables

• Index

• Module Index

• Search Page