מi`.dZddlZddlZddlZddlZddlmZddlmZm Z m Z m Z GddZ de de fd Zd e de fd Zd ede fd Zde d ede fdZy)u scripts/knowledge/kg_to_obsidian.py — JSONL Knowledge Graph → Obsidian Vault converter. Converts Genesis KG entities and axioms (stored as JSONL) into Obsidian-compatible markdown files with YAML frontmatter and [[wiki-link]] cross-references. Usage: from scripts.knowledge.kg_to_obsidian import KGToObsidian converter = KGToObsidian() stats = converter.convert_all() # VERIFICATION_STAMP # Story: 12.02 — KGToObsidian # Verified By: parallel-builder # Verified At: 2026-02-25 # Tests: 9/9 # Coverage: 100% N)Path)DictListOptionalTuplec eZdZdZ ddededdfdZdeeeffdZde eeffdZ de eeffd Z dd Z d e d e d ede eeffdZded ededefdZy) KGToObsidianz Converts Genesis KNOWLEDGE_GRAPH JSONL files into an Obsidian-compatible markdown vault. Each JSONL line becomes one .md file. kg_path vault_pathreturnNcnt||_t||_g|_g|_i|_y)N)rr r _entity_ids _axiom_ids _entity_types)selfr r s 9/mnt/e/genesis-system/scripts/knowledge/kg_to_obsidian.py__init__zKGToObsidian.__init__"s2 G} z*&(%'35c|j\}}|j\}}|j||||zdS)z Run full conversion: entities + axioms. Returns: {"entities": N, "axioms": N, "errors": N} )entitiesaxiomserrors)convert_entitiesconvert_axiomsgenerate_index)r entity_count entity_errors axiom_count axiom_errorss r convert_allzKGToObsidian.convert_all1sN'+&;&;&=# m$($7$7$9! \ $!#l2  rc|jdz }|jdz }|jdd|j||dS)z Iterate all JSONL files in KNOWLEDGE_GRAPH/entities/ and write .md files to vault/entities/. Returns: (converted_count, error_count) rTparentsexist_okentitykindr r mkdir_convert_directory)r entities_in entities_outs rrzKGToObsidian.convert_entitiesAsKllZ/ 3 4$7&&{Lx&PPrc|jdz }|jdz }|jdd|j||dS)z Iterate all JSONL files in KNOWLEDGE_GRAPH/axioms/ and write .md files to vault/axioms/. Returns: (converted_count, error_count) rTr"axiomr&r()r axioms_in axioms_outs rrzKGToObsidian.convert_axiomsNsKLL8+ __x/ 5&&y*7&KKrc|jjddgd}t|jj D]}|j d|j |j dt|j|D]"}t|}|j d|d$|j d|j d|j dt|jD]"}t|}|j d |d$|jd z }|jd j|d y)za Create vault/INDEX.md with wikilinks to all converted entities grouped by type. Tr")u,# Genesis Knowledge Graph — Obsidian Indexz8Auto-generated by `scripts/knowledge/kg_to_obsidian.py`.r2z## Entities by Typer2z### r2z - [[entities/]]z ## Axiomsz - [[axioms/zINDEX.md utf-8encodingN) r r)sortedrkeysappend capitalizesanitize_filenamer write_textjoin)rlinesetypeeidsafeaid index_paths rrzKGToObsidian.generate_index[s9 dT: D..3356 E LL4 0 0 234 5 LL d0078 7(- }TF"56 7 LL     [! R$//* 1C$S)D LL;tfB/ 0 1__z1 dii.Ar source_dirdest_dirr'c d}d}|js||fSt|jD]}|jdvr|j} |j dd}t|jdD]7\} } | j} | s tj| } t| ts|dz }F|j| ||} t!| j#d|d | } t%| d z}||z } |j'| d |dz }|d k(rs|j(j+| t!| j#d | j#dd}|j,j/|gj+| |j0j+| :||fS#t $r |dz }YwxYw#tj$r |dz }YrwxYw#t $r |dz }YwxYw)uu Walk every .jsonl (and .json) file in source_dir. Each line is one record → one .md file. r)z.jsonlz.jsonr5replace)r7r)startidz-linez.mdr6r%typecategoryunknown)existsr8iterdirsuffixname read_textOSError enumerate splitlinesstripjsonloadsJSONDecodeError isinstancedict_record_to_markdownstrgetr<r=rr:r setdefaultr)rrErFr'countr jsonl_file source_namerawlinenolinerecord md_content record_idfilenameout_pathr@s rr*zKGToObsidian._convert_directory~s  "&=  !3!3!56+ 6J  (;;$//K  **GI*N !*#..*:! D 6 zz|!ZZ-F "&$/aKF!55fdKP  4K=fX1N OP ,Y7%?#h.'' W'EQJE 8#$$++I6 66::j)3T UVE&&11%<CCINOO**95A 6+ 6Zf}M !  ++aKF aKFs6GG G6GGG32G36HHrg source_filec g}|jd|jdtt|jddd|dk(rJ|jd|jdd }|jd tt|dnd|jdd }|jd tt|d|jd d}|dk7r|jd||jd|jd|jdd}|r'|jdtt|d|jd|jd|} |jdtt| d|jd|jd|jd|jd|} |jd| |jdt |} | r.t | |} |j| |jd|jdg} | rt | tr|jd|jd| D]f} t | ts| jdd}| jdd}|s;tt|}|jd|d|dh|jd|jdi}|rt |tr|jd |jd|jD]U\}}t |ttfrtj|d!"}n t|}|jd|d#|W|jdd$j|S)%z Convert a single parsed JSON record to Obsidian markdown with YAML frontmatter and [[wiki-link]] cross-references. z---zid: "rKr2"r%rLrMrNztype: "generalz category: " confidencez confidence: created_at timestampdatezdate: "sourcerlzsource_file: "rRz# relationshipsz## Relationshipstarget RELATED_TOz- **u** → [[entities/r3 propertiesz ## PropertiesF) ensure_ascii**: r4)r: _yaml_escaper^r_ _extract_body_inject_wiki_linksr[listr\r<itemsrXdumpsr>)rrgr'rlr?r@rMrpdate_valrttitle body_textrurelrvrel_type safe_targetpropskeyvalval_strs rr]z KGToObsidian._record_to_markdownsF  U vl3vzz$/C+DEFbIJ 8 JJvvzz*i'HIE LL8LU$<#=R@ Azz*i8H LL< S](C'DBG HL"5JR |J<89::  JJ{FJJvr$: ;   LL8LX$?#@C DHfjj &LM |CK'@&ADE U R 66::dK#@A r%\" R"&) *9f=I LL # LL  ?B7 Z t< LL+ , LL $ c4( WWXr2F"wwv|{m2N   LL  <, Zt, LL ) LL !KKM 8ScD$<0"jj5AG!#hG tC5WI67  8 LL yyr)z%/mnt/e/genesis-system/KNOWLEDGE_GRAPHz$/mnt/e/genesis-system/obsidian-vault)r N)__name__ __module__ __qualname____doc__r^rrintr rrrrrr*r\r]rrr r s?@ 6 6 6  6 T#s(^ Q%S/ Q Lc3h LBF::*.:69: sCx:xM M "%M 47M M rr rRr ctjdd|}tjdd|}tjdd|}|jd}|xsdS)z Make a string safe for use as a filename. Replaces spaces and any character that is not alphanumeric, hyphen, underscore, or dot with a hyphen. Strips leading/trailing hyphens. z\s+-z[^A-Za-z0-9._-]z-{2,}z-.unnamed)resubrW)rRs rr<r<sT 66&#t $D 66$c4 0D 66(C &D ::d D  9rvaluec&|jddS)z8Escape double-quotes inside a YAML double-quoted string.rnz\")rH)rs rr{r{ s ==e $$rrgcrdD]8}|j|}|st|ts(|jcSg}hd}|j D]Q\}}||vs t|ts|js-|j d|d|jSdj |S)z} Pull the main prose body from a record. Tries: content, description, axiom, summary, text fields in that order. )content descriptionr.summarytext> rKrsrRrLrtrMrrrprqrxrlrugenesis_actionz**rzz )r_r[r^rWrr:r>)rgfieldrpartsskiprs rr|r|%s Hjj :c3'99;  E D LLN6S d?z#s3 LL2cU$syy{m4 56 ;;u rrctjd}dtjdtfd}|j ||S)z Find ENT-xxx style IDs referenced in text and wrap them as [[wiki-links]]. Only links IDs that look like Genesis entity IDs (ENT- prefix or uppercase hyphenated identifiers). z2\b(ENT-[A-Za-z0-9-]+|AGENT-\d+|AX-[A-Za-z0-9-]+)\bmr cF|jd}t|}d|dS)Nrz [[entities/r3)groupr<)r matched_idrBs r replace_matchz)_inject_wiki_links..replace_matchDs(WWQZ  ,TF"%%r)rcompileMatchr^r)rrg id_patternrs rr}r};s=QRJ&&c& >>- ..r)rrXosrsyspathlibrtypingrrrrr r^r<r{r\r|r}rrrrs& ..k k dCC$%%% $3,/S/$/3/r