documentation to help people contribute

[cantora]

Hello, I'm interested in contributing (specifically, I want to add support for the MC* llvm functionality, especially the disassembler stuff). If possible, could someone add some documentation on how the code generation stuff works (I couldn't find any info on this in the website documentation)? I've been reading through the code in llvmpy/src and llvmpy/gen but there aren't many comments. I'll probably figure it out eventually, but any summary from a core developer of how all this code fits together would really help me a lot. Thanks.

[Siu Kwan Lam]

I am starting to add developers doc at: https://github.com/llvmpy/llvmpy/blob/master/llvmpy/README.md. I hope it has enough to get you started. Let me know if you have any question and I will add to the docs.

Thanks,
Siu

On Aug 1, 2013, at 11:40 PM, cantora [e-mail] wrote:

Hello, I'm interested in contributing (specifically, I want to add support for the MC* llvm functionality, especially the disassembler stuff). If possible, could someone add some documentation on how the code generation stuff works (I couldn't find any info on this in the website documentation)? I've been reading through the code in llvmpy/src and llvmpy/gen but there aren't many comments. I'll probably figure it out eventually, but any summary from a core developer of how all this code fits together would really help me a lot. Thanks.

—
Reply to this email directly or view it on GitHub.

[cantora]

thanks, this is very helpful so far :)

[cantora]

Is there anything I should know about memory management with regard to the bindings? For example, if I create a binding for a method which returns a pointer to a newly allocated object, will the object be freed when the PyObject in which it is encapsulated is freed by the garbage collector? Similarly, if I create a binding to an LLVM method which returns a const pointer to an object that the caller is not obligated to free, will it end up getting freed anyway when the garbage collector frees the PyObject?

The reason I ask is that Im trying to track down a bug where an LLVM object I have seems to be pointing to garbage memory.

[Siu Kwan Lam]

If you define a Destructor, it will be automatically invoked when there are no more reference to the object. But the problem is usually with ownership transfer, you can take a look at the use of ownedptr (https://github.com/llvmpy/llvmpy/search?q=ownedptr&source=c); esp, for Module since its ownership is transferred to the ExecutionEngine. If ownedptr is used in the argument specification, the callee will obtain the ownership. If ownedptr is used in the return value specification, the callee will retain the ownership of the returned value.

[cantora]

thanks for the help.

So ownedptr in a return value specification is sort of a flag which states that the caller which is receiving the object does NOT have to take on the responsibility of freeing the object; i.e. "this pointer I am receiving is owned by someone else".

I you can, please tell me if the below usage is hypothetically correct:

Suppose I have created a binding to the class MCInstrInfo and TargetInstrInfo as such:

MCInstrInfo = llvm.Class()
TargetInstrInfo = llvm.Class(MCInstrInfo)

@MCInstrInfo
class MCInstrInfo:
    _include_ = "llvm/MC/MCInstrInfo.h"

@TargetInstrInfo
class TargetInstrInfo:
    _include_ = 'llvm/Target/TargetInstrInfo.h'

and I also create a binding to the TargetMachine method getInstrInfo:

@TargetMachine
class TargetMachine:
    _include_ = 'llvm/Target/TargetMachine.h'
...
    getInstrInfo = Method(const(ownedptr(TargetInstrInfo)))

Now if I run ii = target_machine.getInstrInfo(); ii = None the garbage collector will destroy the PyObject but the memory pointed to by the return value of getInstrInfo() will not be freed (which is what we want).

On the other hand suppose I have a binding to the Target method createMCInstrInfo:

@Target
class Target:
    getNext = Method(const(ptr(Target)))
...
    createMCInstrInfo = Method(ptr(MCInstrInfo))

Now if I run mcii = target.createMCInstrInfo(); mcii = None the garbage collector will destroy the PyObject and the memory pointed to by the return value of createMCInstrInfo() will be freed since that pointer is not owned by anybody else.

Is this correct?

(BTW: I did not mention anything of destructors here because it seems straightforward that whenever an object is freed, the destructor will be called if it is defined in the binding; my main confusion is when and why an llvm object will be freed in the firstplace.)

[Siu Kwan Lam]

Yes, you are correct.

You can also initiate a pull-request that way we can work together more easily.

[cantora]

ok, thanks for clearing up the memory management stuff. One last question: the above binding of getNext = Method(const(ptr(Target))) in the Target class is not code I wrote, it was there already. From looking at the LLVM docs, it seems like this pointer should not be freed by LLVMPY, so is this a bug? Should it instead be getNext = Method(const(ownedptr(Target)))? or are const(ptr(...)) return specifications implicity owned pointers?

I was planning on sending a pull request as soon as I figured out this invalid object bug, which brings me to my next question: how does inheritance work with the llvmpy binding stuff?

Here's the bug I'm looking at: I have a binding for the MCInstrInfo and TargetInstrInfo classes:

MCInstrInfo = llvm.Class()
TargetInstrInfo = llvm.Class(MCInstrInfo)

@MCInstrInfo
class MCInstrInfo:
    _include_ = "llvm/MC/MCInstrInfo.h"

    get = Method(const(ref(MCInstrDesc)), cast(int, Unsigned))

@TargetInstrInfo
class TargetInstrInfo:
    _include_ = 'llvm/Target/TargetInstrInfo.h'

    #get = Method(const(ref(MCInstrDesc)), cast(int, Unsigned))

and a binding for the getInstrInfo method of the TargetMachine class:

@TargetMachine
class TargetMachine:
    _include_ = 'llvm/Target/TargetMachine.h'

    getInstrInfo = Method(const(ownedptr(TargetInstrInfo)))

The problem is that if I call target_machine.getInstrInfo().get(919), it somehow invokes the get method incorrectly and the this pointer in the context of the get method is wrong. In fact the this pointer is the value of the this pointer for the TargetInstrInfo context of the class instead of the value of the this pointer for the parent MCInstrInfo context of the class (which is supposed to be the context of the call, since TargetInstrInfo does not override the get method).

If I add the binding for the get method in the TargetInstrInfo class (commented above) it works correctly, so is there a way to do inheritance so that it will work without binding the get method in the child?

(btw all this is in this branch: https://github.com/cantora/llvmpy/tree/llvm-mc-3.4)

[Siu Kwan Lam]

ok, thanks for clearing up the memory management stuff. One last question: the above binding of getNext = Method(const(ptr(Target))) in the Target class is not code I wrote, it was there already. From looking at the LLVM docs, it seems like this pointer should not be freed by LLVMPY, so is this a bug? Should it instead be getNext = Method(const(ownedptr(Target)))? or are const(ptr(...)) return specifications implicity owned pointers?

This is likely to be a bug. getNext is also unused.

I was planning on sending a pull request as soon as I figured out this invalid object bug, which brings me to my next question: how does inheritance work with the llvmpy binding stuff?

Here's the bug I'm looking at: I have a binding for the MCInstrInfo and TargetInstrInfo classes:

MCInstrInfo = llvm.Class()
TargetInstrInfo = llvm.Class(MCInstrInfo)

@MCInstrInfo
class MCInstrInfo:
include = "llvm/MC/MCInstrInfo.h"

get = Method(const(ref(MCInstrDesc)), cast(int, Unsigned))

@TargetInstrInfo
class TargetInstrInfo:
include = 'llvm/Target/TargetInstrInfo.h'

#get = Method(const(ref(MCInstrDesc)), cast(int, Unsigned))

and a binding for the getInstrInfo method of the TargetMachine class:

@TargetMachine
class TargetMachine:
include = 'llvm/Target/TargetMachine.h'

getInstrInfo = Method(const(ownedptr(TargetInstrInfo)))

The problem is that if I call target_machine.getInstrInfo().get(919), it somehow invokes the get method incorrectly and the this pointer in the context of the get method is wrong. In fact the this pointer is the value of the this pointer for the TargetInstrInfo context of the class instead of the value of the this pointer for the parent MCInstrInfo context of the class (which is supposed to be the context of the call, since TargetInstrInfo does not override the get method).

If I add the binding for the get method in the TargetInstrInfo class (commented above) it works correctly, so is there a way to do inheritance so that it will work without binding the get method in the child?

I can reproduce the problem. The binding is not doing upcasting properly. I will have to fix that.

[cantora]

OK, glad you can confirm the problem.

regarding the ownedptr stuff, Ill start looking out for bugs like that and fix them as I see them now that I understand how its supposed to work.

[Siu Kwan Lam]

This commit 0801df4 should have fixed the upcasting problem. If you create a patch from it, it should work on your branch.

[cantora]

thanks, your changes fixed all the problems I was having :D