layer slots

[Attila Lendvai]

hi,

while working on a new architecture for verrazano where backends are
layers, i wanted to add slots for layers.

after looking at the contextl code i ended up with two questions:
- there's no public api to get the prototype from the layer-context
on which i could access the slots i've added to my layer
- the prototype in the active layer-context is not instantiated with
make-instance but the class-prototype is used which means that layer
slots are effectively shared among threads (and maybe even after a
nested deactivation/activation of the layer?)

the test files suggest usage like this:

(assert (eq (slot0 (find-layer 'test-layer)) 'foo))

but find-layer also simply calls class-prototype after a find-class on
the layer name, which again is a shared instance. or am i badly
missing here something?

maybe it's ok to call class-prototype because there'll always be a
distinct class? if so, how does it interact with caching?

did i make some sense?

[Attila Lendvai]

Post by Attila Lendvai
while working on a new architecture for verrazano where backends are
layers, i wanted to add slots for layers.

to clarify what i'm trying to achive here: i would like to have slots
on layers that can hold data for "sessions" when the layer is active.

from a different point of view: i would like to represent verrazano
backends (that generate cffi/whatever bindings from gccxml output) as
layers and the layer called 'backend should have a slot called
'output-definition-queue that holds the definitions to be output for a
generation run. then i enable the actual backend layer and i should be
there initialized on the activated layer.

i've added this to my code:

(defun current-layer ()
(contextl::layer-context-prototype (current-layer-context)))

and access the slot on (current-layer), but that is accessing a slot
of the class prototype with all its undesired consequences for my
use-case.

without contextl i would be doing something very similar: introduce a
first argument called backend, make-instance a backend instance and
dispatch on it on all the methods that are now layered methods.

am i misusing layers here?

[Pascal Costanza]

Post by Attila Lendvai

Post by Attila Lendvai
while working on a new architecture for verrazano where backends are
layers, i wanted to add slots for layers.

to clarify what i'm trying to achive here: i would like to have slots
on layers that can hold data for "sessions" when the layer is active.
from a different point of view: i would like to represent verrazano
backends (that generate cffi/whatever bindings from gccxml output) as
layers and the layer called 'backend should have a slot called
'output-definition-queue that holds the definitions to be output for a
generation run. then i enable the actual backend layer and i should be
there initialized on the activated layer.
(defun current-layer ()
(contextl::layer-context-prototype (current-layer-context)))
and access the slot on (current-layer), but that is accessing a slot
of the class prototype with all its undesired consequences for my
use-case.
without contextl i would be doing something very similar: introduce a
first argument called backend, make-instance a backend instance and
dispatch on it on all the methods that are now layered methods.
am i misusing layers here?

I am not sure, but I also have to admit that I don't fully understand
the example yet.

However, what you could do is define a slot in a layer as a special
slot:

(deflayer foo ()
((some-slot :initarg :some-slot :accessor some-slot :special t)))

Such a slot behaves like a special variable: It can have different
bindings in different threads. There is a form in ContextL to activate
a layer and at the same time bind such a slot:

(with-active-layers ((foo :some-slot 'some-value))
...)

This activates 'foo and binds some-slot to 'some-value with dynamic
scope (other threads still see the old value of some-slot and the new
binding will be automatically undone on return from that form).

This kind of layer activation is likely somewhat less efficient, since
it involves looking up initialization keywords and calling
reinitialize-instance on the layer prototypes.

A cheaper solution would actually involve to just use dletf:

(with-active-layers (foo)
(dletf (((some-slot (find-layer 'foo)) 'some-value))
...))

Or maybe just a special variable:

(defvar *some-slot*)

(with-active-layers (foo)
(let ((*some-slot* 'some-value))
...))

;)

I don't know which of those comes closer to solving your concrete
problem.

If none of this does the job, please let me know and I will try to
think harder about a better solution...


Thanks for the feedback,
Pascal

[Attila Lendvai]

Post by Pascal Costanza
I don't know which of those comes closer to solving your concrete
problem.
If none of this does the job, please let me know and I will try to
think harder about a better solution...

Pascal, thanks a lot for the exhaustive answer!

i'm sorry to let you know, but at the end i decided not to use
contextl in verrazano. the rationale is that it gives extra
dependencies and none of the alternative solutions were seamless
enough, so i just added the backend parameters by hand. this way the
resulting code will be more accessible to those who are not familiar
with contextl and i also felt some early overdesign in the air.

i wanted to have numerous layers like struct-emitting,
const-char-pointer-as-string, etc... and a backend layer would have
inherited a number of these. then later on the user could say
additionally an active and an inactive layer list on top of this that
is applied after the backend layer was enabled and before the
generation is started.

and all that while there's only a single cffi backend for verrazano...
so i decided to simply add an extra first backend parameter to the
relevant generics and if someone wants to customize the generation
they can just inherit from a backend and override as necessary.

it doesn't mean that contextl is not useful! but for now i think it
would be too much in verrazano.

[Pascal Costanza]

Post by Attila Lendvai

Post by Pascal Costanza
I don't know which of those comes closer to solving your concrete
problem.
If none of this does the job, please let me know and I will try to
think harder about a better solution...

Pascal, thanks a lot for the exhaustive answer!
i'm sorry to let you know, but at the end i decided not to use
contextl in verrazano. the rationale is that it gives extra
dependencies and none of the alternative solutions were seamless
enough, so i just added the backend parameters by hand. this way the
resulting code will be more accessible to those who are not familiar
with contextl and i also felt some early overdesign in the air.
i wanted to have numerous layers like struct-emitting,
const-char-pointer-as-string, etc... and a backend layer would have
inherited a number of these. then later on the user could say
additionally an active and an inactive layer list on top of this that
is applied after the backend layer was enabled and before the
generation is started.
and all that while there's only a single cffi backend for verrazano...
so i decided to simply add an extra first backend parameter to the
relevant generics and if someone wants to customize the generation
they can just inherit from a backend and override as necessary.
it doesn't mean that contextl is not useful! but for now i think it
would be too much in verrazano.

No need to feel sorry. You should of course only use what actually
contributes to a good solution.

I'd be interested to look at the code once you have it done. It may be
useful to analyze and see how ContextL could have helped if it had the
(yet to be found) right features...

Thanks a lot for consideration!


Pascal

[Attila Lendvai]

Post by Pascal Costanza
No need to feel sorry. You should of course only use what actually
contributes to a good solution.
I'd be interested to look at the code once you have it done. It may be
useful to analyze and see how ContextL could have helped if it had the
(yet to be found) right features...

hm, since then i've dropped a prototype that used contextl and went on
using simple generics.

iirc, the reason was that i couldn't store values in slots of the
current layer and because of that i needed an extra BACKEND argument
in the dispatch. but then anyone can subclass it and add their
customizations to the generation, so i choosed not to add a complex
dependency at the end...

but of course i keep on using contextl in other projects of mine. thanks Pascal!

[Pascal Costanza]

Hi,

Post by Attila Lendvai
hi,
while working on a new architecture for verrazano where backends are
layers, i wanted to add slots for layers.
- there's no public api to get the prototype from the layer-context
on which i could access the slots i've added to my layer

No, but you can always get a prototype for a layer via find-layer. For
example, you can say (find-layer 'foo) for getting the prototype for
layer 'foo.

Post by Attila Lendvai
- the prototype in the active layer-context is not instantiated with
make-instance but the class-prototype is used which means that layer
slots are effectively shared among threads (and maybe even after a
nested deactivation/activation of the layer?)

Yes, layer slots are effectively shared, including across threads.

Layers are actually singletons, and that's intentional. It allows me
to cache layer contexts and do a quick lookup of contexts on each
layer activation/deactivation. If layers weren't singletons, I would
have to create a layer instance for each activation/deactivation,
because I wouldn't know under what other circumstances an already
existing instance is already in use somewhere else. (I could do some
form of layer instance pooling, but that would still be less efficient
than the current scheme.)

When you define slots on layers, as in the following form, these slots
are internally actually turned into class / shared slots.

(deflayer foo ()
((some-slots :accessor some-slot)))

You can access such a slot via the layer prototype: (some-slot (find-
layer 'foo)).

There is always ever one such slot per layer.

Post by Attila Lendvai
(assert (eq (slot0 (find-layer 'test-layer)) 'foo))
but find-layer also simply calls class-prototype after a find-class on
the layer name, which again is a shared instance. or am i badly
missing here something?

No, you found out right.

Post by Attila Lendvai
maybe it's ok to call class-prototype because there'll always be a
distinct class? if so, how does it interact with caching?

The fact that layers are singletons actually enables the caching
scheme of ContextL.

If you really want to create the 'same' kind of layer several times,
you can use subclassing on layers:

(deflayer foo1 (foo))
(deflayer foo2 (foo))
...

In this case, the slots from 'foo are still shared among all those
layers, but you can redefine such slots in sublayers according to the
semantics of overriding shared slots in CLOS.

However, that's probably overkill. See more on that in my next post.


Pascal