# `pwnlib.util.iters` — Extension of standard module `itertools`

This module includes and extends the standard module `itertools`.

pwnlib.util.iters.bruteforce(func, alphabet, length, method='upto', start=None)

  Bruteforce func to return `True`. func should take a string input and return a `bool()`. func will be called with strings from alphabet until it returns `True` or the search space has been exhausted.

  The argument start can be used to split the search space, which is useful if multiple CPU cores are available.

  Parameters:

    - func (function) – The function to bruteforce.

    - alphabet – The alphabet to draw symbols from.

    - length – Longest string to try.

    - method – If ‘upto’ try strings of length `1 .. length`, if ‘fixed’ only try strings of length `length` and if ‘downfrom’ try strings of length `length .. 1`.

    - start – a tuple `(i, N)` which splits the search space up into N pieces and starts at piece i (1..N). `None` is equivalent to `(1, 1)`.

  Returns:

    A string s such that `func(s)` returns `True` or `None` if the search space was exhausted.

  Example

      >>> bruteforce(lambda x: x == 'yes', string.ascii_lowercase, length=5)
      'yes'

pwnlib.util.iters.chained(func)

  A decorator chaining the results of func. Useful for generators.

  Parameters:

    func (function) – The function being decorated.

  Returns:

    A generator function whoose elements are the concatenation of the return values from `func(*args, **kwargs)`.

  Example

      >>> @chained
      ... def g():
      ...     for x in count():
      ...         yield (x, -x)
      >>> take(6, g())
      [0, 0, 1, -1, 2, -2]
      >>> @chained
      ... def g2():
      ...     for x in range(3):
      ...         yield (x, -x)
      >>> list(g2())
      [0, 0, 1, -1, 2, -2]

pwnlib.util.iters.consume(n, iterator)

  Advance the iterator n steps ahead. If n is :const:`None, consume everything.

  Parameters:

    - n (int) – Number of elements to consume.

    - iterator (iterator) – An iterator.

  Returns:

    `None`.

  Examples

      >>> i = count()
      >>> consume(5, i)
      >>> next(i)
      5
      >>> i = iter([1, 2, 3, 4, 5])
      >>> consume(2, i)
      >>> list(i)
      [3, 4, 5]
      >>> def g():
      ...     for i in range(2):
      ...         yield i
      ...         print(i)
      >>> consume(None, g())
      0
      1

pwnlib.util.iters.cyclen(n, iterable) → iterator

  Repeats the elements of iterable n times.

  Parameters:

    - n (int) – The number of times to repeat iterable.

    - iterable – An iterable.

  Returns:

    An iterator whoose elements are the elements of iterator repeated n times.

  Examples

      >>> take(4, cyclen(2, [1, 2]))
      [1, 2, 1, 2]
      >>> list(cyclen(10, []))
      []

pwnlib.util.iters.dotproduct(x, y) → int

  Computes the dot product of x and y.

  Parameters:

    - x (iterable) – An iterable.

    - x – An iterable.

  Returns:

    The dot product of x and y, i.e. – `x[0] * y[0] + x[1] * y[1] + ...`.

  Example

      >>> dotproduct([1, 2, 3], [4, 5, 6])
      ... # 1 * 4 + 2 * 5 + 3 * 6 == 32
      32

pwnlib.util.iters.flatten(xss) → iterator

  Flattens one level of nesting; when xss is an iterable of iterables, returns an iterator whoose elements is the concatenation of the elements of xss.

  Parameters:

    xss – An iterable of iterables.

  Returns:

    An iterator whoose elements are the concatenation of the iterables in xss.

  Examples

      >>> list(flatten([[1, 2], [3, 4]]))
      [1, 2, 3, 4]
      >>> take(6, flatten([[43, 42], [41, 40], count()]))
      [43, 42, 41, 40, 0, 1]

pwnlib.util.iters.group(n, iterable, fill_value=None) → iterator

  Similar to `pwnlib.util.lists.group()`, but returns an iterator and uses `itertools` fast build-in functions.

  Parameters:

    - n (int) – The group size.

    - iterable – An iterable.

    - fill_value – The value to fill into the remaining slots of the last group if the n does not divide the number of elements in iterable.

  Returns:

    An iterator whoose elements are n-tuples of the elements of iterable.

  Examples

      >>> list(group(2, range(5)))
      [(0, 1), (2, 3), (4, None)]
      >>> take(3, group(2, count()))
      [(0, 1), (2, 3), (4, 5)]
      >>> [''.join(x) for x in group(3, 'ABCDEFG', 'x')]
      ['ABC', 'DEF', 'Gxx']

pwnlib.util.iters.iter_except(func, exception)

  Calls func repeatedly until an exception is raised. Works like the build-in `iter()` but uses an exception instead of a sentinel to signal the end.

  Parameters:

    - func (callable) – The function to call.

    - exception (Exception) – The exception that signals the end. Other exceptions will not be caught.

  Returns:

    An iterator whoose elements are the results of calling `func()` until an exception matching exception is raised.

  Examples

      >>> s = {1, 2, 3}
      >>> i = iter_except(s.pop, KeyError)
      >>> next(i)
      1
      >>> next(i)
      2
      >>> next(i)
      3
      >>> next(i)
      Traceback (most recent call last):
          ...
      StopIteration

pwnlib.util.iters.lexicographic(alphabet) → iterator

  The words with symbols in alphabet, in lexicographic order (determined by the order of alphabet).

  Parameters:

    alphabet – The alphabet to draw symbols from.

  Returns:

    An iterator of the words with symbols in alphabet, in lexicographic order.

  Example

      >>> take(8, map(lambda x: ''.join(x), lexicographic('01')))
      ['', '0', '1', '00', '01', '10', '11', '000']

pwnlib.util.iters.lookahead(n, iterable) → object

  Inspects the upcoming element at index n without advancing the iterator. Raises `IndexError` if iterable has too few elements.

  Parameters:

    - n (int) – Index of the element to return.

    - iterable – An iterable.

  Returns:

    The element in iterable at index n.

  Examples

      >>> i = count()
      >>> lookahead(4, i)
      4
      >>> next(i)
      0
      >>> i = count()
      >>> nth(4, i)
      4
      >>> next(i)
      5
      >>> lookahead(4, i)
      10

pwnlib.util.iters.mbruteforce(func, alphabet, length, method='upto', start=None, threads=None)

  Same functionality as bruteforce(), but multithreaded.

  Parameters:

    - func – same as for bruteforce()

    - alphabet – same as for bruteforce()

    - length – same as for bruteforce()

    - method – same as for bruteforce()

    - start – same as for bruteforce()

    - threads – Amount of threads to spawn, default is the amount of cores.

  Example

      >>> mbruteforce(lambda x: x == 'hello', string.ascii_lowercase, length = 10)
      'hello'
      >>> mbruteforce(lambda x: x == 'hello', 'hlo', 5, 'downfrom') is None
      True
      >>> mbruteforce(lambda x: x == 'no', string.ascii_lowercase, length=2, method='fixed')
      'no'
      >>> mbruteforce(lambda x: x == '9999', string.digits, length=4, threads=1, start=(2, 2))
      '9999'

pwnlib.util.iters.nth(n, iterable, default=None) → object

  Returns the element at index n in iterable. If iterable is a iterator it will be advanced.

  Parameters:

    - n (int) – Index of the element to return.

    - iterable – An iterable.

    - default (objext) – A default value.

  Returns:

    The element at index n in iterable or default if iterable has too few elements.

  Examples

      >>> nth(2, [0, 1, 2, 3])
      2
      >>> nth(2, [0, 1], 42)
      42
      >>> i = count()
      >>> nth(42, i)
      42
      >>> nth(42, i)
      85

pwnlib.util.iters.pad(iterable, value=None) → iterator

  Pad an iterable with value, i.e. returns an iterator whoose elements are first the elements of iterable then value indefinitely.

  Parameters:

    - iterable – An iterable.

    - value – The value to pad with.

  Returns:

    An iterator whoose elements are first the elements of iterable then value indefinitely.

  Examples

      >>> take(3, pad([1, 2]))
      [1, 2, None]
      >>> i = pad(iter([1, 2, 3]), 42)
      >>> take(2, i)
      [1, 2]
      >>> take(2, i)
      [3, 42]
      >>> take(2, i)
      [42, 42]

pwnlib.util.iters.pairwise(iterable) → iterator
  Parameters:

    iterable – An iterable.

  Returns:

    An iterator whoose elements are pairs of neighbouring elements of iterable.

  Examples

      >>> list(pairwise([1, 2, 3, 4]))
      [(1, 2), (2, 3), (3, 4)]
      >>> i = starmap(operator.add, pairwise(count()))
      >>> take(5, i)
      [1, 3, 5, 7, 9]

pwnlib.util.iters.powerset(iterable, include_empty=True) → iterator

  The powerset of an iterable.

  Parameters:

    - iterable – An iterable.

    - include_empty (bool) – Whether to include the empty set.

  Returns:

    The powerset of iterable as an interator of tuples.

  Examples

      >>> list(powerset(range(3)))
      [(), (0,), (1,), (2,), (0, 1), (0, 2), (1, 2), (0, 1, 2)]
      >>> list(powerset(range(2), include_empty = False))
      [(0,), (1,), (0, 1)]

pwnlib.util.iters.quantify(iterable, pred=bool) → int

  Count how many times the predicate pred is `True`.

  Parameters:

    - iterable – An iterable.

    - pred – A function that given an element from iterable returns either `True` or `False`.

  Returns:

    The number of elements in iterable for which pred returns `True`.

  Examples

      >>> quantify([1, 2, 3, 4], lambda x: x % 2 == 0)
      2
      >>> quantify(['1', 'two', '3', '42'], str.isdigit)
      3

pwnlib.util.iters.random_combination(iterable, r) → tuple
  Parameters:

    - iterable – An iterable.

    - r (int) – Size of the combination.

  Returns:

    A random element from `itertools.combinations(iterable, r = r)`.

  Examples

      >>> random_combination(range(2), 2)
      (0, 1)
      >>> random_combination(range(10), r = 2) in combinations(range(10), r = 2)
      True

pwnlib.util.iters.random_combination_with_replacement(iterable, r)

  random_combination(iterable, r) -> tuple

  Parameters:

    - iterable – An iterable.

    - r (int) – Size of the combination.

  Returns:

    A random element from `itertools.combinations_with_replacement(iterable, r = r)`.

  Examples

      >>> cs = {(0, 0), (0, 1), (1, 1)}
      >>> random_combination_with_replacement(range(2), 2) in cs
      True
      >>> i = combinations_with_replacement(range(10), r = 2)
      >>> random_combination_with_replacement(range(10), r = 2) in i
      True

pwnlib.util.iters.random_permutation(iterable, r=None)

  random_product(iterable, r = None) -> tuple

  Parameters:

    - iterable – An iterable.

    - r (int) – Size of the permutation. If `None` select all elements in iterable.

  Returns:

    A random element from `itertools.permutations(iterable, r = r)`.

  Examples

      >>> random_permutation(range(2)) in {(0, 1), (1, 0)}
      True
      >>> random_permutation(range(10), r = 2) in permutations(range(10), r = 2)
      True

pwnlib.util.iters.random_product(*args, repeat=1) → tuple
  Parameters:

    - args – One or more iterables

    - repeat (int) – Number of times to repeat args.

  Returns:

    A random element from `itertools.product(*args, repeat = repeat)`.

  Examples

      >>> args = (range(2), range(2))
      >>> random_product(*args) in {(0, 0), (0, 1), (1, 0), (1, 1)}
      True
      >>> args = (range(3), range(3), range(3))
      >>> random_product(*args, repeat = 2) in product(*args, repeat = 2)
      True

pwnlib.util.iters.repeat_func(func, *args, **kwargs) → iterator

  Repeatedly calls func with positional arguments args and keyword arguments kwargs. If no keyword arguments is given the resulting iterator will be computed using only functions from `itertools` which are very fast.

  Parameters:

    - func (function) – The function to call.

    - args – Positional arguments.

    - kwargs – Keyword arguments.

  Returns:

    An iterator whoose elements are the results of calling `func(*args, **kwargs)` repeatedly.

  Examples

      >>> def f(x):
      ...     x[0] += 1
      ...     return x[0]
      >>> i = repeat_func(f, [0])
      >>> take(2, i)
      [1, 2]
      >>> take(2, i)
      [3, 4]
      >>> def f(**kwargs):
      ...     return kwargs.get('x', 43)
      >>> i = repeat_func(f, x = 42)
      >>> take(2, i)
      [42, 42]
      >>> i = repeat_func(f, 42)
      >>> take(2, i)
      Traceback (most recent call last):
          ...
      TypeError: f() takes exactly 0 arguments (1 given)

pwnlib.util.iters.roundrobin(*iterables)

  Take elements from iterables in a round-robin fashion.

  Parameters:

    *iterables – One or more iterables.

  Returns:

    An iterator whoose elements are taken from iterables in a round-robin fashion.

  Examples

      >>> ''.join(roundrobin('ABC', 'D', 'EF'))
      'ADEBFC'
      >>> ''.join(take(10, roundrobin('ABC', 'DE', repeat('x'))))
      'ADxBExCxxx'

pwnlib.util.iters.tabulate(func, start=0) → iterator
  Parameters:

    - func (function) – The function to tabulate over.

    - start (int) – Number to start on.

  Returns:

    An iterator with the elements `func(start), func(start + 1), ...`.

  Examples

      >>> take(2, tabulate(str))
      ['0', '1']
      >>> take(5, tabulate(lambda x: x**2, start = 1))
      [1, 4, 9, 16, 25]

pwnlib.util.iters.take(n, iterable) → list

  Returns first n elements of iterable. If iterable is a iterator it will be advanced.

  Parameters:

    - n (int) – Number of elements to take.

    - iterable – An iterable.

  Returns:

    A list of the first n elements of iterable. If there are fewer than n elements in iterable they will all be returned.

  Examples

      >>> take(2, range(10))
      [0, 1]
      >>> i = count()
      >>> take(2, i)
      [0, 1]
      >>> take(2, i)
      [2, 3]
      >>> take(9001, [1, 2, 3])
      [1, 2, 3]

pwnlib.util.iters.unique_everseen(iterable, key=None) → iterator

  Get unique elements, preserving order. Remember all elements ever seen. If key is not `None` then for each element `elm` in iterable the element that will be rememberes is `key(elm)`. Otherwise `elm` is remembered.

  Parameters:

    - iterable – An iterable.

    - key – A function to map over each element in iterable before remembering it. Setting to `None` is equivalent to the identity function.

  Returns:

    An iterator of the unique elements in iterable.

  Examples

      >>> ''.join(unique_everseen('AAAABBBCCDAABBB'))
      'ABCD'
      >>> ''.join(unique_everseen('ABBCcAD', str.lower))
      'ABCD'

pwnlib.util.iters.unique_justseen(iterable, key=None)

  unique_everseen(iterable, key = None) -> iterator

  Get unique elements, preserving order. Remember only the elements just seen. If key is not `None` then for each element `elm` in iterable the element that will be rememberes is `key(elm)`. Otherwise `elm` is remembered.

  Parameters:

    - iterable – An iterable.

    - key – A function to map over each element in iterable before remembering it. Setting to `None` is equivalent to the identity function.

  Returns:

    An iterator of the unique elements in iterable.

  Examples

      >>> ''.join(unique_justseen('AAAABBBCCDAABBB'))
      'ABCDAB'
      >>> ''.join(unique_justseen('ABBCcAD', str.lower))
      'ABCAD'

pwnlib.util.iters.unique_window(iterable, window, key=None)

  unique_everseen(iterable, window, key = None) -> iterator

  Get unique elements, preserving order. Remember only the last window elements seen. If key is not `None` then for each element `elm` in iterable the element that will be rememberes is `key(elm)`. Otherwise `elm` is remembered.

  Parameters:

    - iterable – An iterable.

    - window (int) – The number of elements to remember.

    - key – A function to map over each element in iterable before remembering it. Setting to `None` is equivalent to the identity function.

  Returns:

    An iterator of the unique elements in iterable.

  Examples

      >>> ''.join(unique_window('AAAABBBCCDAABBB', 6))
      'ABCDA'
      >>> ''.join(unique_window('ABBCcAD', 5, str.lower))
      'ABCD'
      >>> ''.join(unique_window('ABBCcAD', 4, str.lower))
      'ABCAD'

pwnlib.util.iters.chain()

  Alias for `itertools.chain()`.

pwnlib.util.iters.combinations()

  Alias for `itertools.combinations()`

pwnlib.util.iters.combinations_with_replacement()

  Alias for `itertools.combinations_with_replacement()`

pwnlib.util.iters.compress()

  Alias for `itertools.compress()`

pwnlib.util.iters.count()

  Alias for `itertools.count()`

pwnlib.util.iters.cycle()

  Alias for `itertools.cycle()`

pwnlib.util.iters.dropwhile()

  Alias for `itertools.dropwhile()`

pwnlib.util.iters.groupby()

  Alias for `itertools.groupby()`

pwnlib.util.iters.filter()

  Alias for python3-style `filter()`

pwnlib.util.iters.filterfalse()

  Alias for `itertools.filterfalse()`

pwnlib.util.iters.map()

  Alias for python3-style `map()`

pwnlib.util.iters.islice()

  Alias for `itertools.islice()`

pwnlib.util.iters.zip()

  Alias for python3-style `zip()`

pwnlib.util.iters.zip_longest()

  Alias for `itertools.zip_longest()`

pwnlib.util.iters.permutations()

  Alias for `itertools.permutations()`

pwnlib.util.iters.product()

  Alias for `itertools.product()`

pwnlib.util.iters.repeat()

  Alias for `itertools.repeat()`

pwnlib.util.iters.starmap()

  Alias for `itertools.starmap()`

pwnlib.util.iters.takewhile()

  Alias for `itertools.takewhile()`

pwnlib.util.iters.tee()

  Alias for `itertools.tee()`