Giters

deepfire / holotype

An attempt to reimagine knowledge interaction.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

About

We are increasingly lost in the sea of information that surrounds us.

So many valuable things we discover are being routinely lost. So many contexts that ought to, never do intersect. So many thoughts that could, never come.

Why?

Because data entry, interning and recovery is harder than it could be.

Holotype is a general-purpose mind assistant, that is an attempt to change that.

A balanced, varied approach to resolve the conflict between extreme ease of use and completeness of data being captured.

It also is supposed to try to be beautiful to the eye.

This is a design-ish document in disarray (everything in random order), accompanied with some code to support it.

State

Nothing to speak of, yet – mostly just ideas, with some technological groundwork done, like FRP & visualisation: reflex-glfw, and a very basic LambdaCube3d setup for a 2.5D UI.

Goals

This is running in several directions at once, with the obvious caveats applicable.

  1. Map the entire resident context (headspace) available in a convenient way
  2. Facilitate context switching by introducing sub-contexts

Ideas pipeline

Inspiration material

Structure-derived UI, with triple-store-based ontologies
Self-discoverable expert cli interfaces
  • Felix: if you ever wondered what’s the state of the art regarding self-discoverable expert cli interfaces: https://kakoune.org/ <= this editor is crazy good regarding that
Mendix (low-code)
ECS
conceptual intro
apecs
https://github.com/jonascarpay/apecs
ecstasy
http://reasonablypolymorphic.com/blog/why-take-ecstasy
GraphQL
hasura
https://hasura.io
flexbox layout implementations
  • Sketchpad

    Ivan Sutherland’s Sketchpad demo - Object oriented graphics using a constraint based system (https://www.youtube.com/watch?v=6orsmFndx_o)

    Douglas Englebart’s demo - too many innovations to list, but includes real time collaboration (he demoed in a convention center while the system ran 30 miles away in his lab, connected by a leased line operating at 1200 baud!). People think he just invented the mouse, but the overarching theme in his work was augmenting human capabilities… http://dougengelbart.org/firsts/dougs-1968-demo.html

Performance tuning

General random ideas

UI trope: simple is powerful
  1. Arrows move
    • +modifiers = …
  2. Space pages
  3. Tab cycles state/mode
  4. Escape pops
  5. Return vs. C-Return – a story of multi-line text input
UI trope: level-of-detail variance
  1. “give me more data about this object”
    1. scaling
      1. points-of-interest at low scale
    2. Alt-hold-like extension of details shown
  2. sources & foci factoring out common information from data, which is allowed to remain non-enriched – which makes sources+foci be like functions
    • which leads to need for “pinning” values of those projections, for those values the user considered important
Efficient processing of data sets
  1. “well-behaved”: process large datasets efficiently (lazily, if needed)
Principled: mathematical semantics for querying
  1. a multiple set co-reduction/co-projection model
  2. a path language derived from above
Interop
  1. external application embedding (WM-like)
Dealing with external world / state
  1. reify query results as projection called stage, that is out of sync by definition
  2. be very clear about running external processes: can be very frustrating to not know what happens

Toolbox

Open questions

reliable ephemeral identification for tag overlays
How to pin overlaid metadata to source data – there are sources we have no structural (or even mutation) control over, so can’t pin “within” the data.
model does not cover data mutability
Graph representation
DeltaGraph
source
2016 Dexter, Liu, Chau - Lazy Graph Processing in Haskell
conclusion
not ready for consumption, according to authors
data-reify
source
2009 Gill - Type-Safe Observable Sharing in Haskell
conclusion
specific tool for discharging direct object references
Huet’s Zipper
source
2005 Ramsey, Dias - An Applicative Control-Flow Graph Based on Huet’s Zipper
source
2010 Ramsey, Dias, Peyton Jones - Hoopl: A Modular, Reusable Library for Dataflow Analysis and Transformation
key properties
  • unclear improvement over simpler encoding
Lazy I/O and graphs: Winterfell to King’s Landing
source
https://jaspervdj.be/posts/2017-01-17-lazy-io-graphs.html
key properties
  • unsafeInterleaveIO-driven SQL peeking
  • direct object references
The Monad Reader #5 - Practical Graph Handling
source
https://wiki.haskell.org/The_Monad.Reader/Issue5/Practical_Graph_Handling
key properties
Current best idea

A simple map of node ids to nodes.

Interactive development
halive
Data sources
Properties
type
  • structure
  • identification across persistence
  • only for metadata-external types, to enable tag overlays
    rendering
    • meaningful views
    metadata externality
    • local to data sources
    • overlaid from specialized storage
Types
source types
by structure
  • tagged sets
  • hierarchies
    • file system
  • graphs
    element types
    by structure
    atomic
    (point with attributes)
    • meta
    • pdf
    • media
    complex
Storage backends
  • should support rich (schema-capable, version-capable) semantics
Scene composition
Phases, quick overview
Select
filter stores through Selector, yield Selection
Choose presenting engine
emphasize user agency, deemphasize static rules like defaulting
  • context?
Visibility constraint computation
engine decides on how much can be shown
Viewport positioning
engine decides how to place the view around focus
Viewport culling
engine decides on what elements fit into the chosen view
Layout
obtain what is already covered, cover what isn’t, compose; compute scene modifiers
Render
…
Functions, quick overview
select
Structure struc ⇒ Source → Selector struc → Selection struc
compute_cull
Presenter struc eng ⇒ eng → (Granularity, MinSize) → Cull eng
place_viewport
Presenter struc eng ⇒ eng → Selection struc → Focus struc → Cull eng → Viewport eng
cull_selection
Presenter struc eng ⇒ eng → Selection struc → ViewArgs → Viewport eng → (View struc, Boundary eng
layout
Presenter struc eng ⇒ eng → (View struc, Boundary eng) → (Layout eng, Ephemeral eng)
render
RenderContext ren ⇒ ren → (View struc, Boundary eng) → (Layout eng, Ephemeral eng) → IO ()
interact
InputSys is ⇒ is → (View struc, Boundary eng) → Affective → Affective
Phases
Select
Source → Selector → Selection
What
select from Source
  • Selections split into the following categories, by structure:
    • General graph
    • DAG (directed acyclic graph)
    • Set – with customisable ordering
      • XXX: ordering not factored in
Design considerations
  • XXX: live-updating selections
    • just carry update frequency for re-selection? (DONE)
    • any kind of policy that would be more.. reactive?
  • XXX: partial selections?
    • what for?
      • for hopelessly large data sets we can limit
        • but a dumb cutoff isn’t useful
        • so, a smart, movable cutoff is needed
  • does it make sense for a selector to be non-specific about what it returns?
    • hard to say just yet, we need experience as guide
Presenter choice
Selection → PresPref → Presenter
  • PresPref picks a specific Presenter, compatible with the current Selection structure:
    • defaults to last used
    • size limits for non-partial-capable engines?
    • can be cycled through by a shortcut
  • Engines:
    • Graph, dag, tree:
      • SideGraph: graph from aside
      • DownGraph: graph, arrow aligned weighted partitioning
    • Dag (duplicates-encoded), tree:
      • DagList, list entries
      • DagGrid, icon grid
      • DagSpace, space partitioning, ala Lamdu
    • Set:
      • Carousel
      • Grid
      • List
  • Summing up, fundamentally we want:
    1. type classes for individual LEs, because it allows for a seriously neat organisation of code
    2. multiple LEs associated with a structure, because that’s how the problem domain looks
    3. #1 gives that there isn’t a monotype for a LE
Visibility constraints computation
Presenter → (Granularity, MinSize) → Cull
  • disconnected from specific elements – deals with UI constraints projected onto a specific layout engine:
    • for SideGraph and SideDag – no idea, let practice guide us..
    • for space partitioning it’s trivial – granularity says it all
    • for a Grid and DagGrid – how many rows and columns
    • for a List – how many rows
  • updated only rarely – when the user changes the visualisation parameters
Viewport positioning
Presenter → Selection → Focus → Cull → Viewport
  • How do we position a viewport?
    • If we don’t have a focus, then it wouldn’t make sense to have a viewport
      • Pick a “first” element (maybeHead $ fromList set, e.g.)
    • If we don’t have a viewport, generate one containing the focus
    • if we do have a viewport, and the focus is inside – choice is upon the engine
    • if we the focus is outside, shift the viewport – how exactly is upon the engine
  • The above exposes following questions:
    • what does “inside a viewport” mean?
    • how can we generate a viewport that is guaranteed to contain a focus?
  • The answer seems to have the shape of a structure-specific visibility constraint specifier – a Cull.
Viewport culling
Presenter → Selection → (Granularity, MinSize) → Viewport → (View, Boundary)
  • XXX: what’s the story about half-visible objects?
    • select all intersecting, render more than what is showable?
  • XXX: what’s the story about avoidable layout recomputation?
    • key question: is it bad? In case of SideGraph, which is about total representation, it’s very very bad.
    • caseanalysis cacheable total-cost can-partial partial-composable
      • SideGraph: yes very hard no(?) no(?)
      • DownGraph: no medium-small yes yes
      • DagList: yes small yes yes
      • DagGrid: yes medium-small yes yes
      • DagSpace: yes very hard yes yes
      • Carousel: no easy no no
      • Grid: yes easy yes yes
      • List: yes easy yes yes
    • option: compute base layout, then viewportcull and localise from base
      • for huge selections this produces unnecessary computation
    • option: go with partials and compose them, whenever possible
      • if so, layout needs to be:
        • restartable at arbitrary point
        • splittable and composable
    • option: lazy evaluation?
    • NOTE: all obvious caching solutions seem to rely on Ord
  • Granularity determines, for tree layouts, the maximum depth of subdivision, after which abbreviation is engaged
  • MinSize limits the minimum element size
  • Viewport is specific to Presenter:
    • SideGraph: layout-global position
    • DownGraph: subroot node
    • DagList: row offset
    • DagGrid: row offset
    • DagSpace: vertical offset (it’s possible, because it’s weighted space partitioning, but…?)
    • Carousel: current selection
    • Grid: row offset
    • List: row offset
  • View is direct elements from Selection
  • Boundary is anchor points to the parts of Selection that fall outside the Viewport
Layout
Presenter → (View, Boundary) → (Layout, Ephemerals)
  • XXX: Positions what are they?
    • scene-specific structure and interpretation?
    • if not, global or screenspace?
    • pixel-based, or [0.0..1.0]?
  • Ephemerals are inherently non-persistent, layout-specific things like:
    • element focus visulalisation state:
      • scale change, to indicate foreground/background
Change summary
  • What effect did the last Selector change have? Not always obvious.
Render
RenderContext → (View, Boundary) → (Layout, Ephemerals) → IO ()
Interaction
Inputs → (View, Boundary) → Focus → (Granularity, MinSize) → Selector → PresPref → (Modifiers, Focus, (Granularity, MinSize), Selector, PresPref)

Concepts available for implementation

(To be) Displayable structures
Graphs
Views
Z-axis
Classic side view
Needs root detection, for automatic layout.
Arrow-aligned
Weighted partitioning
Dags
Views
Z-axis
inherited from Graphs
Y-axis
inherited from Graphs
Treeview, list entries, with duplication
Treeview, icon grid, with duplication
Treeview, space partitioning, ala Lamdu, with duplication
Subsetting
Viewport
Arrow walker – for nodes. Iterative refinement – subsetting and context narrowing. Some kind of a shortcut-based jump language. Bookmarks.
Ellipsis
Zoomable: “everything else in this direction” What cases need it, given a proper Viewport subsetter?
Sets
Views
Carousel
Grid
List
Subsetting
Viewport
Iterative refinement makes it useful. Arrow walker – for refinement elements and for.
Summary
Extracting and exposing set structure.
Ellipsis
Logic summary or an explicit summary.
Visualisable qualities
Exhaustivity
  • Explicit “unknown” remaining
Variant-ness
  • Simultaneous
  • Per-choice filtering
Progression
Distinctions
  • Decomposition vs. dependency

Implemented

Substrates

Flatland – primordial soup of units of scale, color, dimension, area

..and associated operations

HoloCairo – Flatland + Cairo-based drawing & font-work
  • fairly flexible font selection with aliases, vector/bitmap distinction, variants and defaulting

Components

Flex – 2d Flexbox-based abstract layout engine
HoloPort – manage/show abstract Visuals with identity
  • LambdaCube3D-based, so richly extensible
  • picking supported
  • 2.5D
  • screen/frame management
  • targetable by HoloCairo
Holo – Reflex FRP-based composable widget layer
  • vocabulary:
    As
    a Name that Denotes a type
    Interp
    Interpret a type into another
    Mutable
    evolution in response to events subscribed to
    Holo
    build upon the above – mix input events with others to define a dynamic As/Interp-defined interactive widget
Considerations
  • why did we (mistakenly) go with: (As a, As b) => As (a, b)
    • originally: -> Holo (Di a)
    • also: Denoted n ~ (a, a)
  • the mistake of (As a, As b) => As (a, b) – necessitates own, intra-widget focus management, since specialised input is impossible due to genericity
  • let’s go back to generic Holo (Di a)
    • does it need an As n, Denoted n ~ a, Interp a (Di b)?
    • What is implementable/not for a multi-Identity composite?
      • [-] As n, Denoted n ~ Composite – necessitates a single Identity
      • [-] Mutable Composite – we have a multitude of identities and want to reuse generic focus machinery
      • [X] Named Composite b
      • [X] Interp Composite b
    • prerequisite lift step doable generically via liftWRecord on (,)
  • let’s turn liftWRecord into a Holo instance?
    • ..would require As
    • -> impossible?
SOP.Monadic – abstract applicative+monadic operation on SOPs
  • generic: monadically recovers a datatype from the structure of a related datatype, with relationship treated in applicative context
  • allows us to lift single-product records into editable widgets
Goal & its problems: liftW = liftWRecord
  1. t & m that liftWRecord depends on are ambiguous
    1. must be somehow deduced from the Holo’s head-bound vars

Playbook

GHC compiler going AWOL

  • -ddump-tc-trace
  • -dcore-lint

Open question archive

flex notes

Attributes
  • width, height ∷ float – absolute-only?
  • left, right, top, bottom ∷ float – def(0), ???
  • padding_LRTB, margin_LRTB ∷ float – def(0)
  • justify_content ∷ def(ALIGN_START)
  • align_content ∷ def(ALIGN_STRETCH)
  • align_items ∷ def(ALIGN_START)
  • align_self ∷ def(ALIGN_AUTO)
  • position ∷ def(POSITION_RELATIVE)
  • direction ∷ def(DIRECTION_COLUMN)
  • wrap ∷ def(NO_WRAP)
  • grow ∷ def(0)
  • shrink ∷ def(1)
  • order ∷ def(0)
  • basis ∷ def(0)
flex_item
  • …attributes (see above) ∷ xxx
  • frame ∷ float[4]
  • parent ∷ ptr flex_item
  • children ∷ [ptr flex_item]
  • should_order_children ∷ bool
flex_layout
  • set during init
    • wrap ∷ bool
    • reverse ∷ bool – whether main axis is reversed
    • reverse2 ∷ bool – whether cross axis is reversed (wrap only)
    • vertical ∷ bool
    • size_dim ∷ float – main axis parent size
    • align_dim ∷ float – cross axis parent size
    • frame_pos_i ∷ uint – main axis position
    • frame_pos2_i ∷ uint – cross axis position
    • frame_size_i ∷ uint – main axis size
    • frame_size2_i ∷ uint – cross axis size
    • ordered_indices ∷ [int]
  • set for each line layout
    • line_dim ∷ float – the cross-axis size
    • flex_dim ∷ float – the flexible part of the main axis size
    • flex_grows ∷ int
    • flex_shrinks ∷ int
    • pos2 ∷ float – cross axis position
    • lines ∷ [struct flex_layout_line]
      • child_begin ∷ uint
      • child_end ∷ uint
      • size ∷ float
    • lines_count ∷ uint
    • lines_sizes ∷ float
Function index
  • update_should_order_children() ∷ set parent’s should_order_children to true
  • item_property_changed(property) ∷ property ≡ order → update_should_order_children
  • flex_item_new/free() ∷ malloc + default attributes & stuff / free() children, then self

*******

  • grow_if_needed ∷ flex_item → void
  • child_set ∷ flex_item → flex_item → int → void
  • flex_item_add ∷ flex_item → flex_item → void
  • flex_item_insert ∷ flex_item → void
  • flex_item_delete ∷ flex_item → flex_item
  • flex_item_count ∷ flex_item → uint
  • flex_item_child ∷ flex_item → flex_item
  • flex_item_parent ∷ flex_item → flex_item
  • flex_item_root ∷ flex_item → flex_item
  • flex_item_get_framex,y,width,height ∷ flex_item → float

*******

  • layout_init ∷ flex_item → float → float → flex_layout → void let width/height = args.w/args.h - item→padding_left - item→padding_right (,,,,) reverse vertical size_dim align_dim frame_pos{,2}_i frame_size{,2}_i = case item→direction of DIRECTION_ROW_REVERSE | f width height DIRECTION_ROW | DIRECTION_COLUMN_REVERSE | DIRECTION_COLUMN | ordered_indices = | f item→should_order_children
    item→children_count
    item→children
    – sorted children indices by their .order property

    flexdim,grows,shrinks = (,,) 0 0 0 wrap = item→wrap != NO_WRAP (,) pos2 reverse2 = | f wrap item→wrap

    align_dim
    vertical
    item→padding_top

    lines = [] in Layout{..}

  • layout_cleanup ∷ flex_layout → void

*******

  • LAYOUT_RESET ∷ flex_layout → flex_layout layout & line_dim .~ if wrap then 0 else align_dim & flex_dim .~ size_dim & flex_grows .~ 0 & flex_shrinks .~ 0
  • _LAYOUT_FRAME ∷ layout → child → {pos,pos2,size,size2} → float
  • CHILD_POS, CHILD_POS2, CHILD_SIZE, CHILD_SIZE2 = _LAYOUT_FRAME(…)
  • CHILD_MARGIN ∷ child → if_vertical ∷ bool → if_horizontal ∷ bool →

*******

  • layout_align ∷ align ∷ flex_align → flex_dim ∷ float → children_count ∷ uint → pos_p ∷ ptr float → spacing_p ∷ ptr float → stretch_allowed ∷ bool
  • child_align ∷ child ∷ flex_item → parent ∷ flex_item → flex_align
  • layout_items ∷ item ∷ flex_item → child_begin ∷ uint → child_end ∷ uint → children_count ∷ int → layout ∷ flex_layout → void
  • layout_item ∷ item ∷ flex_item → width ∷ float → height ∷ float
  • flex_layout ∷ item ∷ flex_item → void
Algorithm

Layout data flow summary

Note: this leaves view-porting (as an overflow handling mechanism) out of scope for now.

Possibilities
LeafModifierFromTopStyleHardnessToTopShrink methodNotes
TextOne-line()font, unbreakHardAbsno
()font, unbreakSoftRelellipsis
BreakableAbsCstrfontSoftRelellipsishard breakable is ⊥
ImageFixed()fixedHardAbsnosoft fixed image is ⊥, unless viewporting
ScalableAbsCstr()SoftRelscale
???any other leaf types?
InterModifierFromTopStyleFromBotToBotHardness HonoringToTopNotes
BoxAbsCstr()Abs/Rel?Rebalancing
WrapAbsCstrthicknessAbs/Rel??????????????????

Apparent fallout from fundamentals ∷

  1. Hard requirements are naturally context-free
  2. Relative hards are possible, though (ratios being a question of design)
  3. Context-ful requirements are impossible up-front, in a single pass

Observations ∷

  1. Child ratio knowledge is minimum for Box’s downward propagation of AbsCstr
  2. Some children don’t have ratios, but absolutes can be relativised (absolutisation of relatives is a feasible dual that can lead to better pixel-level stability)
  3. #1 + #2 → child ratios always available, and always immediately – assuming no inter-level balancing

Box hardness honoring procedure ∷

  1. Query all children for direct requirements
  2. Allocate hards (absolutising relatives), computing the remaining soft share
  3. Relativise all soft absolutes from #1
  4. When softs sum to overflow, normalise them
  5. When softs sum to underflow, normalise them, unless there are filler children
  6. Absolutise softs back
  7. When there’s underflow and fillers, distrubute slack between fillers
  8. ??? hards overflow handling policy
    • ideally, propagate upward
      • as a “lacks absolute N”?

Summary: hards first, then redistribute remainder while keeping fillers in mind.

Wrap hardness honoring procedure ∷

  1. Query children for direct requirements
  2. Absolutes that fit exactly: easy
  3. Absolutes that underflow: ???
  4. Absolutes that overflow:
    • ideally, propagate upward (see same for box hardness)

Problems

Node finalisation needs the internal type

Potential solution
  1. Make Item sport the content type by default, only wrapping it for children.
  2. Bonus: make Item into an HList-like structure!
    • that has a risk of making refactoring super-painful in the current prototyping phase, though.

(m :.: Result t) a ⇒ Applicative on ‘Result t’ ⇒ Node construction unbearably pure

The problematic pipeline
  1. ct = (!! choice) $ SOP.apInjs_POP pop ∷ SOP (m :.: Result t) xss
  2. By hsequence ∷ (SListIN h xs, SListIN (Prod h) xs, HSequence h, Applicative f) => h f xs -> f (h I xs)
  3. Comp msop = hsequence ct ∷ (m :.: Result t) (SOP I xss)
  4. By unComp ∷ (f :.: g) p -> f (g p)
  5. msop ∷ m (Result t (SOP I xss))
  6. By (SOP.to <$>) <$> m (Result t (SOP I (x : xs))) -> m (Result t a)
  7. Comp res = (SOP.to <$>) <$> msop ∷ (m :.: Result t) a
  8. By unComp ∷ (f :.: g) p -> f (g p)
  9. res ∷ m (Result t a)
Potential `hsequence` replacements
  • hsequence’ ∷ (SListIN h xs, Applicative f) ⇒ h (f :.: g) xs → f (h g xs) hsequence ∷ h f xs → f (h I xs) htraverse’ ∷ (SListI2 xss, Applicative g) => (forall a ⇒ f a → g (f’ a)) → SOP f xss → g (SOP f’ xss)
  • g ~ m, f’ ~ Result t
    • (forall a ⇒ f a → m (Result t a)) → SOP f xss → m (SOP (Result t) xss)
The problematic foundation of the pipeline
  1. forall f xs. (c a, All c xs, All2 c xss) ⇒ … → (a → f) → (m :.: Result t) f
  2. We want to separate (Result t f) from f
  3. Theory: can we go for this instead: → m (Result t f)
  4. Let’s try!
Solution
  • introduce a separate node finalisation post-phase, that performs monadic computation after the ADT lift

GHC-8.8

  • cabal-doctest
    • [X] fetchpatch
  • cabal-install
    • [X] bump for Cabal
  • generic-deriving
    • [X] bump for TH
  • data-default
    • [ ] ???
      • [“cp”, “-f”, ”nix/store/baa1q5ph9w2daw63vdblldq4wc5g74c0-data-default-class-0.1.2.0/lib/ghc-8.7.20190115/package.conf.d/data-default-class-0.1.2.0-FeIQ5tLoVZBHMSgrT9zptQ.conf”, “/build/package.conf.d”]
      • [“cp”, “-f”, ”build/package.conf.d”]
  • haskell-src-exts
    • [ ] sigsegv in Haddock
  • gtk2hs-buildtools
  • th-lift
  • generics-sop

emacs options

;; Local Variables: ;; eval: (setf indent-tabs-mode nil org-todo-keyword-faces ‘((“TODO” . “#6c71c4”) (“START” . “#2aa198”) (“CODE” . “#6c71c4”) (“SORTA” . “#268bd2”) (“DONE” . “#073642”) (“UPSTREAM” . “#268bd2”))) ;; End:

About

An attempt to reimagine knowledge interaction.

License:GNU Affero General Public License v3.0

Languages

Language:Haskell 96.7%Language:Nix 1.4%Language:Makefile 1.4%Language:Shell 0.3%Language:C 0.1%Language:Emacs Lisp 0.0%