The aim of this work is to classify visualization verification reports by the requirement they violate. In other words: “Don’t just tell me that the visualization is broken, tell me how to fix it.”

veriviz verifies block diagrams visualized in XGEE. XGEE visualizes models based on a visualization model that specifies how model elements are visualized as vertices, edges, and labels. When the visualization is correct, we can be trust that “what you see is what you get”. This allows further usage of the model without manual verification of the visualization. veriviz leverages the same visualization model in the reverse direction. While XGEE maps model to visualization, veriviz recognizes the model from the visualization. veriviz performs recognition in the stages: tokenization, syntactical analysis, and instantiation. Recognition is followed by a comparison step that compares the recognized model to the original model. These four stages can deliver findings, e.g., tokenization can report that unknown artifacts are in the screenshot. The combined findings form a report. Usually, the report is empty and confirms a correct visualization. When an error is introduced, domain experts can typically diagnose the cause and know how to fix. This motivates the research question: Can veriviz reports be classified such that non-experts can resolve visualization problems?

The space of possible visualization errors is defined by the set of DO-330 Tool Operational Requirements. Therefore, requirements of veriviz have been refined from rough categories into a precise catalog of more than 50 requirements. For example, the requirement “VIS shall visualize elements according to the visualization model” is refined to “VIS shall color vertices according to the visualization model”. Requirements are tested by a reference visualization expected to yield no findings and a dedicated test case in which exactly that requirement is violated (e.g., a vertex rendered in green instead of blue). Test cases have been created in at least one, and often two, visualization models (Open Avionics Architecture Model: Functions and Hardware). Findings from the four veriviz stages are consolidated into eight finding types, augmented with contextual attributes, e.g., the area of untokenized pixels. For every test case, the count of individual finding types forms a classification feature vector. The vector is extended with handcrafted features that are as well independent from the visualization model. They encode expert knowledge by aggregating multiple findings, e.g., “unknown artifacts close to missing parent”. This yields a ground-truth mapping from requirement violations to feature vectors, the violation mapping catalog.

For previously unseen block diagrams, the report is converted into a feature vector and matched against the catalog. First, exact matches are searched. Second, k-nearest neighbors based on L1 distance. The output is the most likely predicted violated requirements together with a preconfigured recovery recommendation. Preliminary results indicate that already a few handcrafted features lead to feature vectors with distinctive signatures. Ongoing experiments evaluate the degree of visualization model independence and the classification performance of new block diagrams.