High-level Algorithm
Encoder Algorithm

## Topics

| Speed vs Quality Trade Off
| Source Frame Processing
| Rate Control
| Temporal Dependency Modelling
| Two Pass Mode
| The Look-Ahead Buffer
| Golden Frame Group

## Functions

int | av1_pack_bitstream (struct AV1_COMP *const cpi, uint8_t *dst, size_t dst_size, size_t *size, int *const largest_tile_id)
| Pack the bitstream for one frame.
int | av1_encode_strategy (AV1_COMP *const cpi, size_t *const size, uint8_t *const dest, size_t dest_size, unsigned int *frame_flags, int64_t *const time_stamp, int64_t *const time_end, const aom_rational64_t *const timestamp_ratio, int *const pop_lookahead, int flush)
| Implement high-level encode strategy.
static void | encode_frame_internal (AV1_COMP *cpi)
| Encoder setup(only for the current frame), encoding, and recontruction for a single frame.
void | av1_encode_frame (AV1_COMP *cpi)
| Setup reference frame buffers and encode a frame.
static void | cdef_restoration_frame (AV1_COMP *cpi, AV1_COMMON *cm, MACROBLOCKD *xd, int use_restoration, int use_cdef, unsigned int skip_apply_postproc_filters)
| Select and apply cdef filters and switchable restoration filters.
static void | loopfilter_frame (AV1_COMP *cpi, AV1_COMMON *cm)
| Select and apply deblocking filters, cdef filters, and restoration filters.
static int | encode_without_recode (AV1_COMP *cpi)
| Encode a frame without the recode loop, usually used in one-pass encoding and realtime coding.
static int | encode_with_recode_loop (AV1_COMP *cpi, size_t *size, uint8_t *dest, size_t dest_size)
| Recode loop for encoding one frame. the purpose of encoding one frame for multiple times can be approaching a target bitrate or adjusting the usage of global motions.
static int | encode_with_recode_loop_and_filter (AV1_COMP *cpi, size_t *size, uint8_t *dest, size_t dest_size, int64_t *sse, int64_t *rate, int *largest_tile_id)
| Recode loop or a single loop for encoding one frame, followed by in-loop deblocking filters, CDEF filters, and restoration filters.
static int | encode_frame_to_data_rate (AV1_COMP *cpi, size_t *size, uint8_t *dest, size_t dest_size)
| Run the final pass encoding for 1-pass/2-pass encoding mode, and pack the bitstream.
int | av1_receive_raw_frame (AV1_COMP *cpi, aom_enc_frame_flags_t frame_flags, const YV12_BUFFER_CONFIG *sd, int64_t time_stamp, int64_t end_time_stamp)
| Obtain the raw frame data.
int | av1_get_compressed_data (AV1_COMP *cpi, AV1_COMP_DATA *const cpi_data)
| Encode a frame.
int | av1_encode (AV1_COMP *const cpi, uint8_t *const dest, size_t dest_size, const EncodeFrameInput *const frame_input, const EncodeFrameParams *const frame_params, size_t *const frame_size)
| Run 1-pass/2-pass encoding.

## Detailed Description

This module describes sequence level/frame level algorithm in AV1. More details will be added.

## Function Documentation

## av1_encode_strategy()

int av1_encode_strategy | ( | AV1_COMP *const | cpi,
|  | size_t *const | size,
|  | uint8_t *const | dest,
|  | size_t | dest_size,
|  | unsigned int * | frame_flags,
|  | int64_t *const | time_stamp,
|  | int64_t *const | time_end,
|  | const aom_rational64_t *const | timestamp_ratio,
|  | int *const | pop_lookahead,
|  | int | flush )

Implement high-level encode strategy.

This function will implement high-level encode strategy, choosing frame type, frame placement, etc. It populates an EncodeFrameParams struct with the results of these decisions and then encodes the frame. The caller should use the output parameters *time_stamp and *time_end only when this function returns AOM_CODEC_OK.

Parameters

  [in] | cpi | Top-level encoder structure
  [out] | size | Bitstream size
  [out] | dest | Bitstream output buffer
  [in] | dest_size | Bitstream output buffer size
  [in] | frame_flags | Flags to decide how to encoding the frame
  [out] | time_stamp | Time stamp of the frame
  [out] | time_end | Time end
  [in] | timestamp_ratio | Time base
  [in] | pop_lookahead | Decide to pop the source frame from queue
  [in] | flush | Decide to encode one frame or the rest of frames

Returns
  Returns a value to indicate if the encoding is done successfully.

Return values

  AOM_CODEC_OK |
  -1 |
  AOM_CODEC_ERROR |

References AOM_CBR, AOM_CODEC_ERROR, AOM_CODEC_OK, AOM_EFLAG_FORCE_KF, AOM_Q, AOM_RC_FIRST_PASS, AOM_RC_ONE_PASS, AOM_RC_SECOND_PASS, AOM_RC_THIRD_PASS, av1_cyclic_refresh_update_parameters(), av1_encode(), av1_get_second_pass_params(), av1_lookahead_depth(), av1_lookahead_peek(), av1_lookahead_pop_sz(), av1_set_rtc_reference_structure_one_layer(), RATE_CONTROL::base_frame_target, AV1_COMP::common, AV1_COMP::compressor_stage, RateControlCfg::cq_level, AV1Common::cur_frame, AV1Common::current_frame, RateControlCfg::drop_frames_water_mark, AV1_COMP::droppable, AV1Common::error, AV1_COMP::ext_flags, AV1Common::features, AV1Common::film_grain_params, AV1_COMP::film_grain_table, TimeStamps::first_ts_start, AV1Common::frame_presentation_time, EncodeFrameParams::frame_type, AV1_COMP::framerate, AV1_COMP::gf_frame_index, AV1_PRIMARY::gf_group, has_no_stats_stage(), AV1Common::height, AV1_COMP::is_dropped_frame, KeyFrameCfg::key_freq_max, AV1EncoderConfig::kf_cfg, AV1_PRIMARY::lookahead, AV1Common::mi_params, RateControlCfg::mode, AV1_COMP::mv_stats, AV1_PRIMARY::mv_stats, AV1_COMP::orig_source, AV1_COMP::oxcf, AV1_PRIMARY::p_rc, AV1EncoderConfig::pass, AV1_COMP::ppi, TimeStamps::prev_ts_end, AV1Common::quant_params, PRIMARY_RATE_CONTROL::rate_correction_factors, AV1_COMP::rc, AV1EncoderConfig::rc_cfg, EncodeFrameParams::ref_frame_flags, ExternalFlags::ref_frame_flags, AV1Common::ref_frame_map, AV1_COMP::ref_refresh_index, AV1_COMP::refresh_frame, EncodeFrameParams::refresh_frame, ExternalFlags::refresh_frame, AV1_COMP::refresh_idx_available, AV1Common::remapped_ref_idx, EncodeFrameParams::remapped_ref_idx, SPEED_FEATURES::rt_sf, AV1_PRIMARY::rtc_ref, AV1_COMP::second_pass_log_stream, AV1Common::seq_params, AV1_COMP::sf, AV1_PRIMARY::show_existing_alt_ref, EncodeFrameParams::show_frame, AV1_COMP::skip_tpl_setup_stats, AV1_COMP::source, EncodeFrameParams::speed, AV1_COMP::svc, AV1_COMP::td, AV1_COMP::time_stamps, AV1_PRIMARY::tpl_data, TplParams::tpl_stats_pool, AV1_PRIMARY::twopass, AV1_COMP::twopass_frame, AV1_COMP::unfiltered_source, AV1_COMP::unscaled_source, ExtRefreshFrameFlagsInfo::update_pending, AV1_PRIMARY::use_svc, CommonQuantParams::using_qmatrix, and AV1Common::width.

Referenced by av1_get_compressed_data().

## av1_encode_frame()

void av1_encode_frame | ( | AV1_COMP * | cpi | ) |

Setup reference frame buffers and encode a frame.

Parameters

  [in] | cpi | Top-level encoder structure

References AV1_COMP::alloc_pyramid, FeatureFlags::allow_warped_motion, AV1_COMP::common, AV1Common::cur_frame, AV1Common::current_frame, AV1_COMP::enc_seg, encode_frame_internal(), AV1Common::features, HIGH_LEVEL_SPEED_FEATURES::frame_parameter_update, SPEED_FEATURES::hl_sf, FeatureFlags::interp_filter, CommonTileParams::large_scale, EncSegmentationInfo::map, CommonModeInfoParams::mi_cols, AV1Common::mi_params, CommonModeInfoParams::mi_rows, AV1_COMP::oxcf, AV1_COMP::palette_pixel_num, FeatureFlags::reduced_tx_set_used, AV1_COMP::ref_frame_dist_info, AV1_COMP::ref_frame_flags, SPEED_FEATURES::rt_sf, AV1Common::seg, AV1_COMP::sf, AV1_COMP::source, FeatureFlags::switchable_motion_mode, AV1_COMP::td, AV1Common::tiles, and FeatureFlags::tx_mode.

Referenced by encode_with_recode_loop(), and encode_without_recode().

## encode_without_recode()

int encode_without_recode | ( | AV1_COMP * | cpi | ) |

static

Encode a frame without the recode loop, usually used in one-pass encoding and realtime coding.

Parameters

  [in] | cpi | Top-level encoder structure

Returns
  Returns a value to indicate if the encoding is done successfully.

Return values

  AOM_CODEC_OK |
  AOM_CODEC_ERROR |

References AV1EncoderConfig::algo_cfg, AV1_COMP::alloc_pyramid, AOM_CBR, AOM_CODEC_MEM_ERROR, AOM_CODEC_OK, av1_cyclic_refresh_setup(), av1_encode_frame(), av1_encodedframe_overshoot_cbr(), av1_set_speed_features_qindex_dependent(), av1_set_variance_partition_thresholds(), FeatureFlags::byte_alignment, AV1_COMP::common, AV1_COMP::consec_zero_mv, AV1_COMP::consec_zero_mv_alloc_size, AV1Common::cur_frame, AV1Common::current_frame, SVC::downsample_filter_phase, SVC::downsample_filter_type, RateControlCfg::drop_frames_water_mark, aom_roi_map::enabled, AV1_COMP::enc_quant_dequant_params, AV1Common::error, FeatureFlags::error_resilient_mode, AV1Common::features, AV1_COMP::gf_frame_index, AV1_PRIMARY::gf_group, RefreshFrameInfo::golden_frame, AV1Common::height, SPEED_FEATURES::inter_sf, AV1_COMP::last_source, CommonModeInfoParams::mi_cols, AV1Common::mi_params, CommonModeInfoParams::mi_rows, RateControlCfg::mode, AV1_COMP::orig_source, AV1_COMP::oxcf, SPEED_FEATURES::part_sf, AV1_COMP::ppi, AV1Common::prev_frame, FeatureFlags::primary_ref_frame, AV1Common::quant_params, AV1_COMP::rc, AV1EncoderConfig::rc_cfg, AV1_COMP::ref_frame_flags, AV1_COMP::refresh_frame, AV1_COMP::roi, SPEED_FEATURES::rt_sf, AV1_PRIMARY::rtc_ref, AV1_COMP::scaled_last_source, AV1_COMP::scaled_last_source_available, AV1_COMP::scaled_source, AV1Common::seg, AV1Common::seq_params, AV1_COMP::sf, AlgoCfg::sharpness, AV1_COMP::source, AV1_COMP::svc, AV1_COMP::unscaled_last_source, AV1_COMP::unscaled_source, AV1_PRIMARY::use_svc, and AV1Common::width.

Referenced by encode_with_recode_loop_and_filter().

## encode_with_recode_loop()

int encode_with_recode_loop | ( | AV1_COMP * | cpi,
|  | size_t * | size,
|  | uint8_t * | dest,
|  | size_t | dest_size )

static

Recode loop for encoding one frame. the purpose of encoding one frame for multiple times can be approaching a target bitrate or adjusting the usage of global motions.

Parameters

  [in] | cpi | Top-level encoder structure
  [in] | size | Bitstream size
  [out] | dest | Bitstream output buffer
  [in] | dest_size | Bitstream output buffer size

Returns
  Returns a value to indicate if the encoding is done successfully.

Return values

  AOM_CODEC_OK |
  -1 |
  AOM_CODEC_ERROR |

References AV1EncoderConfig::algo_cfg, AV1_COMP::alloc_pyramid, FeatureFlags::allow_high_precision_mv, AOM_CODEC_ERROR, AOM_CODEC_OK, AOM_Q, AOM_RC_THIRD_PASS, AOM_SUPERRES_NONE, av1_encode_frame(), av1_pack_bitstream(), av1_set_speed_features_qindex_dependent(), av1_set_variance_partition_thresholds(), RATE_CONTROL::coefficient_size, AV1_COMP::common, AV1Common::cur_frame, AV1Common::current_frame, AV1Common::delta_q_info, HIGH_LEVEL_SPEED_FEATURES::disable_extra_sc_testing, AV1_COMP::do_frame_data_update, AV1_COMP::ducky_encode_info, AV1_COMP::enc_quant_dequant_params, AV1Common::error, AV1_COMP::ext_ratectrl, AV1Common::features, AV1_COMP::gf_frame_index, AV1_PRIMARY::gf_group, AV1_COMP::gm_info, AV1Common::height, HIGH_LEVEL_SPEED_FEATURES::high_precision_mv_usage, SPEED_FEATURES::hl_sf, InterpSearchFlags::interp_filter_search_mask, AV1_COMP::interp_search_flags, SPEED_FEATURES::interp_sf, AV1_COMP::last_source, CommonModeInfoParams::mi_cols, AV1Common::mi_params, CommonModeInfoParams::mi_rows, RateControlCfg::min_cr, RateControlCfg::mode, AV1_COMP::mv_stats, AV1_COMP::num_frame_recode, NUM_RECODES_PER_FRAME, AV1_COMP::oxcf, SPEED_FEATURES::part_sf, AV1EncoderConfig::pass, AV1_COMP::ppi, AV1Common::prev_frame, RATE_CONTROL::projected_frame_size, AV1Common::quant_params, AV1_COMP::rc, AV1EncoderConfig::rc_cfg, AV1_COMP::rd, HIGH_LEVEL_SPEED_FEATURES::recode_loop, recode_loop_update_q(), AV1_COMP::scaled_last_source, AV1_COMP::scaled_source, GlobalMotionInfo::search_done, AV1Common::seg, AV1Common::seq_params, AV1_COMP::sf, AlgoCfg::sharpness, AV1_COMP::source, AV1_COMP::superres_mode, AV1Common::superres_scale_denominator, AV1_COMP::unscaled_last_source, AV1_COMP::unscaled_source, AV1_COMP::use_ducky_encode, and AV1Common::width.

Referenced by encode_with_recode_loop_and_filter().

## encode_with_recode_loop_and_filter()

int encode_with_recode_loop_and_filter | ( | AV1_COMP * | cpi,
|  | size_t * | size,
|  | uint8_t * | dest,
|  | size_t | dest_size,
|  | int64_t * | sse,
|  | int64_t * | rate,
|  | int * | largest_tile_id )

static

Recode loop or a single loop for encoding one frame, followed by in-loop deblocking filters, CDEF filters, and restoration filters.

Parameters

  [in] | cpi | Top-level encoder structure
  [in] | size | Bitstream size
  [out] | dest | Bitstream output buffer
  [in] | dest_size | Bitstream output buffer size
  [in] | sse | Total distortion of the frame
  [in] | rate | Total rate of the frame
  [in] | largest_tile_id | Tile id of the last tile

Returns
  Returns a value to indicate if the encoding is done successfully.

Return values

  AOM_CODEC_OK |
  AOM_CODEC_ERROR |

References FeatureFlags::allow_intrabc, AV1_COMP::ambient_err, AOM_CODEC_ERROR, AOM_CODEC_OK, av1_pack_bitstream(), av1_postencode_drop_cbr(), CommonQuantParams::base_qindex, RATE_CONTROL::coefficient_size, AV1_COMP::common, AV1Common::cur_frame, AV1_COMP::do_update_frame_probs_interpfilter, AV1_COMP::do_update_frame_probs_obmc, AV1_COMP::do_update_frame_probs_txtype, AV1_COMP::do_update_frame_probs_warp, AV1_COMP::do_update_vbr_bits_off_target_fast, AV1_COMP::ducky_encode_info, encode_with_recode_loop(), encode_without_recode(), AV1Common::features, RATE_CONTROL::frames_to_key, SPEED_FEATURES::hl_sf, loopfilter_frame(), AV1_COMP::mt_info, NUM_RECODES_PER_FRAME, AV1_COMP::oxcf, AV1_PRIMARY::p_rc, MultiThreadInfo::pipeline_lpf_mt_with_enc, AV1_COMP::ppi, AV1Common::quant_params, AV1_COMP::rc, AV1_COMP::rd, HIGH_LEVEL_SPEED_FEATURES::recode_loop, AV1Common::render_height, AV1Common::render_width, AV1_PRIMARY::rtc_ref, AV1Common::seq_params, AV1_COMP::sf, AV1_COMP::source, and AV1_COMP::use_ducky_encode.

Referenced by encode_frame_to_data_rate().

## encode_frame_to_data_rate()

int encode_frame_to_data_rate | ( | AV1_COMP * | cpi,
|  | size_t * | size,
|  | uint8_t * | dest,
|  | size_t | dest_size )

static

Run the final pass encoding for 1-pass/2-pass encoding mode, and pack the bitstream.

Parameters

  [in] | cpi | Top-level encoder structure
  [in] | size | Bitstream size
  [out] | dest | Bitstream output buffer
  [in] | dest_size | Bitstream output buffer size

Returns
  Returns a value to indicate if the encoding is done successfully.

Return values

  AOM_CODEC_OK |
  AOM_CODEC_ERROR |

References AV1EncoderConfig::algo_cfg, FeatureFlags::allow_ref_frame_mvs, FeatureFlags::allow_screen_content_tools, FeatureFlags::allow_warped_motion, AOM_CBR, AOM_CODEC_ERROR, AOM_CODEC_MEM_ERROR, AOM_CODEC_OK, AOM_RC_THIRD_PASS, av1_pack_bitstream(), CommonQuantParams::base_qindex, AlgoCfg::cdf_update_mode, RATE_CONTROL::coefficient_size, AV1_COMP::common, AV1Common::cur_frame, FeatureFlags::cur_frame_force_integer_mv, AV1Common::current_frame, AV1Common::current_frame_id, FeatureFlags::disable_cdf_update, AV1_COMP::do_frame_data_update, AV1_COMP::ducky_encode_info, encode_with_recode_loop_and_filter(), AV1_COMP::existing_fb_idx_to_show, AV1Common::fc, AV1Common::features, AV1_COMP::force_intpel_info, AV1_COMP::frames_since_last_update, AV1_COMP::gf_frame_index, AV1_PRIMARY::gf_group, has_no_stats_stage(), AV1Common::height, AV1_COMP::is_dropped_frame, AV1EncoderConfig::kf_cfg, CommonTileParams::large_scale, AV1Common::last_frame_seg_map, AV1_COMP::last_frame_type, AV1_COMP::last_show_frame_buf, AV1Common::lf, RateControlCfg::mode, AV1_COMP::num_tg, AV1_COMP::oxcf, AV1EncoderConfig::pass, AV1_COMP::ppi, AV1Common::quant_params, AV1_COMP::rc, AV1EncoderConfig::rc_cfg, AV1_COMP::rd, AV1Common::ref_frame_id, FeatureFlags::refresh_frame_context, SPEED_FEATURES::rt_sf, AV1_PRIMARY::rtc_ref, AV1Common::seg, AV1Common::seq_params, AV1_COMP::sf, KeyFrameCfg::sframe_dist, AV1Common::show_frame, CommonTileParams::single_tile_decoding, AV1_COMP::source, AV1_COMP::superres_mode, AV1_COMP::svc, RateControlCfg::target_bandwidth, AV1_COMP::td, AV1_COMP::tile_data, AV1Common::tiles, AV1_COMP::unscaled_last_source, AV1_COMP::use_ducky_encode, and AV1Common::width.

Referenced by av1_encode().

## av1_receive_raw_frame()

int av1_receive_raw_frame | ( | AV1_COMP * | cpi,
|  | aom_enc_frame_flags_t | frame_flags,
|  | const YV12_BUFFER_CONFIG * | sd,
|  | int64_t | time_stamp,
|  | int64_t | end_time_stamp )

Obtain the raw frame data.

This function receives the raw frame data from input.

Parameters

  [in] | cpi | Top-level encoder structure
  [in] | frame_flags | Flags to decide how to encoding the frame
  [in,out] | sd | Contain raw frame data
  [in] | time_stamp | Time stamp of the frame
  [in] | end_time_stamp | End time stamp

Returns
  Returns a value to indicate if the frame data is received successfully.

Note
  The caller can assume that a copy of this frame is made and not just a copy of the pointer.

< Y (Luminance) plane

< Y (Luminance) plane

References AV1_COMP::alloc_pyramid, AOM_BITS_10, AOM_CODEC_ERROR, AOM_CODEC_INVALID_PARAM, AOM_PLANE_Y, AOM_RC_FIRST_PASS, av1_lookahead_push(), AV1_COMP::common, AV1Common::error, AV1_PRIMARY::lookahead, AV1_COMP::oxcf, AV1EncoderConfig::pass, AV1_COMP::ppi, and AV1Common::seq_params.

## av1_get_compressed_data()

int av1_get_compressed_data | ( | AV1_COMP * | cpi,
|  | AV1_COMP_DATA *const | cpi_data )

Encode a frame.

This function encodes the raw frame data, and outputs the frame bit stream to the designated buffer. The caller should use the output parameters cpi_data->ts_frame_start and cpi_data->ts_frame_end only when this function returns AOM_CODEC_OK.

Parameters

  [in] | cpi | Top-level encoder structure
  [in,out] | cpi_data | Data corresponding to a frame encode

Returns
  Returns a value to indicate if the encoding is done successfully.

Return values

  AOM_CODEC_OK |
  -1 | No frame encoded; more input is required.
  A nonzero (positive) aom_codec_err_t code | The encoding failed with the error. Sets the error code and error message in `cpi->common.error`.

References AOM_CODEC_ERROR, AOM_CODEC_OK, av1_encode_strategy(), av1_one_pass_cbr_svc_start_layer(), AV1_COMP::available_bs_size, CommonQuantParams::base_qindex, AV1_COMP::common, AV1_COMP::compressor_stage, AV1_COMP::counts, AV1Common::current_frame, AV1_COMP_DATA::cx_data, AV1_COMP_DATA::cx_data_sz, AV1Common::error, AV1Common::features, AV1_COMP_DATA::flush, AV1_COMP_DATA::frame_size, AV1_COMP::gf_frame_index, AV1_PRIMARY::gf_group, AV1_COMP::is_dropped_frame, AV1_COMP_DATA::lib_flags, AV1_COMP::oxcf, AV1EncoderConfig::pass, AV1_COMP_DATA::pop_lookahead, AV1_COMP::ppi, AV1Common::quant_params, AV1_COMP::rc, FeatureFlags::refresh_frame_context, AV1Common::seq_params, AV1Common::show_existing_frame, AV1Common::show_frame, AV1Common::showable_frame, AV1_COMP::td, AV1_COMP_DATA::timestamp_ratio, AV1_COMP_DATA::ts_frame_end, AV1_COMP_DATA::ts_frame_start, and AV1_PRIMARY::use_svc.