Skip to main content

CIL Verification and Safety

I've lamented here and elsewhere some unfortunate inconveniences and asymmetries in the CLR -- for example, we have nullable structs but lack non-nullable reference types, an issue I address in my Sasa class library.

I've recently completed some Sasa abstractions for safe reflection, and an IL rewriter based on Mono.Cecil which allows C# source code to specify type constraints that are supported by the CLR but unnecessarily restricted in C#. In the process, I came across another unjustified decision regarding verification: the jmp instruction.

The jmp instruction strikes me as potentially incredibly useful for alternative dispatch techniques, and yet I recently discovered that it's classified as unverifiable. This seems very odd, since the instruction is fully statically typed, and I can't think of a way its use could corrupt the VM.

In short, the instruction performs a control transfer to a named method with a signature matching exactly the current method's signature, as long as the evaluation stack is empty and you are not currently in a try-catch block (see section 3.37 of the ECMA specification).

This seems eminently verifiable given a simple control-flow analysis, an analysis which the verifier already performs to verify control-flow safety of some other verifiable instructions. If anyone can shed some light on this I would appreciate it.

Comments

Rodrigo Kumpera said…
You forgot to mention an additional restriction required to make jmp verifiable. No managed pointers must be passed as arguments.

It would otherwise require much more sophisticated verification rules.

Besides possible historical artifacts and mistakes, one issue is that some implementation specific constraints could make it impossible to implement jmp. And unlike the tail prefix, it doesn't allow for conditional execution.

Honestly, I never got it, the need for jmp. A tail call can do the exect same thing and be JIT'd to the same code sequence.
Sandro Magi said…
I don't think I understand your objection. What do you consider a "managed pointer"?

If you just meant an ordinary reference, then I disagree that this is unsafe.

If you meant an actual unsafe pointer, well we're already handling unsafe types so verification isn't an issue.

I agree that general tail calls are better, but MS is kind of stubborn that tail calls remain inefficient. JMP just seemed like a sort of middle ground that might be useful in lieu of tail calls for some scenarios.

Finally, I disagree that it doesn't allow for conditional execution. All that's required is that the evaluation stack is empty and you're not in a protected block. This should be more or less valid:
IL_001: ldnull
IL_002: brtrue IL_004
IL_003: jmp void Target1(void)
IL_004: jmp void Target2(void)
Rodrigo Kumpera said…
A managed pointer is 335 parlance for a byref argument/local.

Tail calls/jmp to methods with such kind of parameters are not verifiable without complex dataflow analysis or significant restrictions.

I did use a very bad wording for 'conditional execution' of tail calls. I meant that a CLI implementation is not required to obey a tail prefix, while it must for jmp.
Sandro Magi said…
Right, passing a byref argument to a local should be prevented. It's pretty trivial to do this conservatively, ie. detect when there's both a jmp and an starg of a local address in the same function.

A control-flow analysis to detect whether the starg and the jmp are ever on the same execution path is just a nice refinement.
Qwertie said…
I agree, not only would a jmp be useful for exotic dispatch, but often (in performance-sensitive code) I write two versions of a method, a "public" version that verifies the arguments and a "protected" one that assumes the arguments are valid:

void RemoveAt(int i)
{ /* verify the index and call RemoveAtCore */ }
void RemoveAtCore(int i)

This allows code inside the class to bypass the index check if the caller knows it's safe. A tail call helps, but a jmp might be more efficient since you know the JIT won't duplicate the argument(s).

It is irritating also that tail calls cannot be guaranteed to work, so self-recursive functions can cause stack overflow in cases where the equivalent code in Haskell couldn't.

Anyway, surely the verifier should at least allow jmp in cases where there are no "ref" args (given that it would be so little work to implement.)

Popular posts from this blog

Blue-eyed Islander Puzzle - an analysis

Many people find themselves stumped by the so-called Blue-Eyed Islanders puzzle . There is also much controversy over its supposed solution. I'm going to analyze the problem and the solution, and in the process, explain why the solution works. To begin, let's modify the problem slightly and say that there's only 1 blue-eyed islander. When the foreigner makes his pronouncement, the blue-eyed islander looks around and sees no other blue eyes, and being logical, correctly deduces that his own eyes must be blue in order for the foreigner's statement to make sense. The lone blue-eyed islander thus commits suicide the following day at noon. Now comes the tricky part, and the source of much confusion. Let's say there are 2 blue-eyed islanders, Mort and Bob. When the foreigner makes his pronouncement, Mort and Bob look around and see only each other. Mort and Bob thus both temporarily assume that the other will commit suicide the following day at noon. Imagine their chagrin...

Easy Reverse Mode Automatic Differentiation in C#

Continuing from my last post on implementing forward-mode automatic differentiation (AD) using C# operator overloading , this is just a quick follow-up showing how easy reverse mode is to achieve, and why it's important. Why Reverse Mode Automatic Differentiation? As explained in the last post, the vector representation of forward-mode AD can compute the derivatives of all parameter simultaneously, but it does so with considerable space cost: each operation creates a vector computing the derivative of each parameter. So N parameters with M operations would allocation O(N*M) space. It turns out, this is unnecessary! Reverse mode AD allocates only O(N+M) space to compute the derivatives of N parameters across M operations. In general, forward mode AD is best suited to differentiating functions of type: R → R N That is, functions of 1 parameter that compute multiple outputs. Reverse mode AD is suited to the dual scenario: R N → R That is, functions of many parameters t...

Extensible, Statically Typed Pratt Parser in C#

I just completed a statically typed Pratt-style single-state extensible lexer+parser, otherwise known as a top-down operator precedence parser, for the Sasa library . The implementation is available in the Sasa.Parsing dll, under Sasa.Parsing.Pratt . Two simple arithmetic calculators are available in the unit tests . This implementation is novel in two ways: Aside from an alleged implementation in Ada , this is the only statically typed Pratt parser I'm aware of. Pratt parsers typically require a pre-tokenized input before parsing semantic tokens, but I've eliminated this step by using the symbol definitions to drive a longest-match priority-based lexer. Symbols by default match themselves, but you can optionally provide a scanning function used to match arbitrary string patterns. The symbol selected for the current position of the input is the symbol that matches the longest substring. If two symbols match equally, then the symbol with higher precedence is selected. The design...