text-graph-generalization

Code for our paper Compositional Generalization with Grounded Language Models
git clone https://esimon.eu/repos/text-graph-generalization.git
Log | Files | Refs | README | LICENSE

README.md (2628B)

      1 <h2 align="center"><b>Compositional Generalization with Grounded Language Models</h2><br></b>
      2 
      3 <p align="center">
      4   <b>Sondre Wold, Étienne Simon, Lucas Georges Gabriel Charpentier, Egor V. Kostylev, Erik Velldal, Lilja Øvrelid</b>
      5 </p>
      6 
      7 <p align="center">
      8   <i>
      9     University of Oslo<br>
     10   </i>
     11   <br>
     12 </p>
     13 <br>
     14 <p align="center">
     15   <a href=""><b>Paper</b></a><br>
     16 </p>
     17 
     18 _______
     19 
     20 ### Downloding and preprocessing the data
     21 
     22 Details on how to generate the raw LUBM graphs can be found in
     23 `download_data.md`
     24 
     25 ### Sampling
     26 
     27 Sampling the pairs of entities in the graphs that are later translated into
     28 the questions is done through `graph_sampler.py`.
     29 
     30 The sampler takes as input the raw graphs produced by the LUBM UBA and
     31 outputs two things: a folder containing graph objects in `graphml` format,
     32 and a folder that has a `jsonl` file that contains all the dataset samples
     33 for the GNN encoder. 
     34 
     35 E.g, in order to produce all the dataset samples for 2 hops, you can run the
     36 following:
     37 
     38 ```
     39 python graph_sampler.py
     40     --graph-output-path data/2_hop_graphs/
     41     --question-output-path data/2_hop_questions/
     42     --input-path raw_graphs
     43     -k 2
     44     -n 30000
     45 ```
     46 
     47 To produce the dataset samples for 3 and 4 hops, run the same command but
     48 swap the folder names and the `k` argument accordingly.
     49 
     50 ### Creating the dataset splits
     51 Run `dataset.py` with `--task` being either `sys`, `subst` or `prod` to
     52 create the datasets used for the three experiments from the paper.
     53 
     54 ### Running the experiments
     55 `train.py` runs all the experiments on the provided dataset splits. An axplantion of the CLI can be found in the `argpase` method in this file. The file runs the experiments over a set of seeds
     56 provided by a white space seperated list to `--seed`.  
     57 
     58 The different models are selected by setting the following model-specific
     59 flags:
     60 - Baseline (gnn-only): `--gnn`
     61 - Disjoint: `--static`
     62 - Grounded: no flag
     63 - Unidirectional: `--unidirectional`
     64 - Bidirectional: `--unidirectional --bidirectional`
     65 
     66 The hyperparameters we used for the experiments in the paper can be found
     67 in Appendix B.
     68 
     69 ### Requirements
     70 
     71 The experiments ran using the following specific packages, with Python
     72 `3.9.12`:
     73 
     74 ```
     75 jsonlines==4.0.0
     76 networkx==3.2.1
     77 numpy==1.26.3
     78 safetensors==0.4.2
     79 tokenizers==0.15.1
     80 torch==1.11.0+rocm4.5.2
     81 torch-cluster==1.6.3
     82 torch-scatter==2.1.2
     83 torch-sparse==0.6.18
     84 torch-spline-conv==1.2.2
     85 torch_geometric==2.4.0
     86 transformers==4.37.1
     87 ```
     88 
     89 ### Results
     90 Results are logged to Wandb and written to a json file.  The `--debug`
     91 flags disables Wandb logging.
     92 You can use ```write_results.py``` to aggreate the results across seeds and
     93 all models.