Skip to content
Open
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions InternalDocs/interpreter.md
Original file line number Diff line number Diff line change
Expand Up @@ -506,6 +506,68 @@ After the last `DEOPT_IF` has passed, a hit should be recorded with
After an optimization has been deferred in the adaptive instruction,
that should be recorded with `STAT_INC(BASE_INSTRUCTION, deferred)`.

## How to add a new bytecode specialization
Comment thread
hugovk marked this conversation as resolved.

Assuming you found an instruction that serves as a good specialization candidate.
Let's use the example of [`CONTAINS_OP`](../Doc/library/dis.rst#contains_op):
Comment thread
hugovk marked this conversation as resolved.
Outdated

1. Update below in [Python/bytecodes.c](../Python/bytecodes.c)
Comment thread
hugovk marked this conversation as resolved.
Outdated

- Convert `CONTAINS_OP` to a micro-operation (uop) by renaming
it to `_CONTAINS_OP` and changing the instruction definition
from `inst` to `op`.
Comment thread
hugovk marked this conversation as resolved.
Outdated

```c
// Before
inst(CONTAINS_OP, ...);

// After
op(_CONTAINS_OP, ...);
```

- Add a uop that calls the specializing function:

```c
specializing op(_SPECIALIZE_CONTAINS_OP, (counter/1, left, right -- left, right)) {
#if ENABLE_SPECIALIZATION
if (ADAPTIVE_COUNTER_IS_ZERO(counter)) {
next_instr = this_instr;
_Py_Specialize_ContainsOp(right, next_instr);
Comment thread
hugovk marked this conversation as resolved.
Outdated
DISPATCH_SAME_OPARG();
}
STAT_INC(CONTAINS_OP, deferred);
DECREMENT_ADAPTIVE_COUNTER(this_instr[1].cache);
#endif /* ENABLE_SPECIALIZATION */
}
```

- Create a macro for the original bytecode name:
Comment thread
hugovk marked this conversation as resolved.
Outdated

```c
macro(CONTAINS_OP) = _SPECIALIZE_CONTAINS_OP + _CONTAINS_OP;
```

2. Define the cache structure in [Include/internal/pycore_code.h](../Include/internal/pycore_code.h),
at the very least, a 16-bit counter is needed.
Comment thread
hugovk marked this conversation as resolved.
Outdated

```c
typedef struct {
uint16_t counter;
} _PyContainsOpCache;
```

3. Write the specializing function itself (`_Py_Specialize_ContainsOp`) in [Python/specialize.c ](../Python/specialize.c).
Comment thread
hugovk marked this conversation as resolved.
Outdated
Refer to other functions in that file for the pattern.

4. Add a call to `add_stat_dict` in `_Py_GetSpecializationStats` which is in [Python/specialize.c ](../Python/specialize.c).
Comment thread
hugovk marked this conversation as resolved.
Outdated

5. Add the cache layout in [Lib/opcode.py](../Lib/opcode.py) so that Python's
`dis` module will know how to represent it properly.
Comment thread
hugovk marked this conversation as resolved.
Outdated

6. Bump magic number in [Include/core/pycore_magic_number.h](../Include/internal/pycore_magic_number.h).
Comment thread
hugovk marked this conversation as resolved.
Outdated

7. Run ``make regen-all`` on `*nix` or `build.bat --regen` on Windows.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like the only thing missing is adding an actual specialized variant. Maybe that's implied/obvious, but it wouldn't hurt to provide a dumb example of _Py_Specialize_ContainsOp and _CONTAINS_OP_UNICODE_UNICODE that just guards and calls PyUnicode_Contains or something.


Additional resources
--------------------
Expand Down