COEST TraceLab Challenge at SST’19: Instructions

The goal of the COEST TraceLab Challenge is to help familiarize students, researchers, and practitioners with TraceLab in a friendly, fun setting. Participants will compose TraceLab components into an experiment to address a challenge provided by an outside expert. All contributions will be evaluated and a winner will be announced during SST’19.

What you should do before you come to SST:

  • Familiarize yourself with TraceLab
    • Watch this video that shows how to use TraceLab
    • Install TraceLab by following these instructions for Visual Studio 2017 or these instructions for Visual Studio 2019
    • Run your first experiment (VectorSpaceStandardExperiment.teml) that is provided with TraceLab
      • Open TraceLab
      • Select the folder icon
      • Select the COEST directory on your computer
      • Select TraceLab (under the COEST folder)
      • Select Tutorials (under the TraceLab folder)
      • Select First experiment under that folder
      • Select VectorSpaceStandardExperiment.teml
      • Press the green arrow at top left to run the experiment
      • Press on the “I” icon for each component and read about the components, look at their inputs/outputs, configuration, etc.
      • Look at the workspace to see what data structures are used
      • Look at the results generated by the experiment
    • Look at the component tab and see all the components that come with TraceLab, play with these by dragging them into an experiment and running it
  • Find more TraceLab components that you can add to your directory (under COEST, TraceLab, Components)
  • Make up your own experiment challenges, then compose components to solve that challenge.

Day of the Challenge

  • Read the challenge given to you – it will be something such as “Trace the CCHIT source artifact to the target artifact using the information retrieval technique of your choosing with Porter stemming and Fox stopword removal, calculate recall, precision, and f-measure”
  • Compose existing components to solve the challenge
  • (Optionally) Write your own component to use in the challenge
  • Run the experiment
  • Repeat above steps until you are satisfied with your results
  • Provide the results to a judge
  • Have fun!!!

The Judges

The results of the TraceLab challenge are going to be judged by three internationally reknowned experts:

The winner will be announced after lunch and will receive a commemorative plaque!

Acknowledgements

…and we thank Carlos Eduardo Bernal Cardenas of William and Mary for
developing the similarity-matrix-subset and similarity-matrix-transitive-merger components that will be used in the
TraceLab Challenge