9 Supplement to the literature review

This appendix contains the code used to fetch the literature review data using scopus API as well as the prompt used to categorize the papers.

9.1 Scopus search retrieval

require 'typhoeus'
require 'json'
require 'bibtex'
require 'active_support'
require 'active_support/core_ext'


API_KEY = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
API_URL = "https://api.elsevier.com/"
DOI_BASE  = "https://doi.org/"

DEFAULT_OPTIONS = {
  method: :get,
  headers: {
    'Accept': 'application/json',
    'X-ELS-APIKey': API_KEY
  }
}

BIBTEX_TYPES = {
  "Conference Paper" => :inproceedings,
  "Article" => :article,
  "Editorial" => :article,
  "Note" => :article,
  "Review" => :article,
  "Chapter" => :inbook,
  "Article in Press" => :article,
  "Short Survey" => :article,
  "Book" => :book
}

@search_string = 'KEY ("computational creativity")'
@bib = BibTeX::Bibliography.new
@urls = []

def make_keyword_request(sstring, cursor = 0)
  options = DEFAULT_OPTIONS.dup
  options[:params] = {
    query: sstring,
    field: "prism:url",
    count: 25,
    start: cursor
  }
  request = Typhoeus::Request.new(api("content/search/scopus"), options)
  request.on_complete do |response|
    if response.success?
      parsed_response = JSON.parse(response.body)["search-results"]
      if parsed_response["entry"]
        parsed_response["entry"].each do |result|
          puts result["prism:url"]
          @urls.push result["prism:url"]
        end
        make_keyword_request(sstring, cursor + 25).run
      end
    elsif response.timed_out?
      # aw hell no
      puts "got a time out"
    elsif response.code == 0
      # Could not get an http response, something's wrong.
      puts response.return_message
    else
      # Received a non-successful http response.
      puts "HTTP request failed: " + response.code.to_s
      puts response.body
    end
  end
  request
end

def api(path)
  API_URL + path
end


def make_abstract_request(url)
  puts url
  options = DEFAULT_OPTIONS.dup
  options[:params] = {
    view: "META_ABS",
    apiKey: API_KEY
  } 
  request = Typhoeus::Request.new(url, options)
  request.on_complete do |response|
    if response.success?
      parsed_response = JSON.parse(response.body)["abstracts-retrieval-response"]
      coredata = parsed_response["coredata"]
      puts "Found #{coredata["subtypeDescription"]}" 
      puts coredata 
      if is_valid_record(coredata)
        puts "Valid record"
        entry = {}
        entry[:bibtex_type] = BIBTEX_TYPES[coredata["subtypeDescription"]]
        entry[:issn] = coredata["prism:issn"]
        entry[:journal] = coredata["prism:publicationName"]
        entry[:author] = format_authors(coredata["dc:creator"]["author"])
        entry[:abstract] = coredata["dc:description"]
        date = Date.parse(coredata["prism:coverDate"])
        entry[:year] = date.year
        entry[:month] = date.month
        entry[:title] = coredata["dc:title"]
        entry[:volume] = coredata["prism:volume"]
        entry[:number] =  coredata["issueIdentifier"]
        entry[:pages] = "#{coredata["prism:startingPage"]}--#{coredata["prism:endingPage"]}"
        entry[:citedby] = coredata["citedby-count"]
        if coredata["prism:doi"]
          entry[:doi] = coredata["prism:doi"]
          entry[:url] = DOI_BASE + coredata["prism:doi"]
        end
        entry[:eid] = coredata["eid"]
        
        entry[:publisher] = coredata["dc:publisher"]
        
        #entry[:file] = build_file_url(coredata["dc:identifier"])
        if parsed_response["authkeywords"]
          if parsed_response["authkeywords"]["author-keyword"].instance_of? Array
            entry[:keywords] =  parsed_response["authkeywords"]["author-keyword"].map {|key| key["$"].titlecase.strip }.join(", ")
          else
            entry[:keywords] =  parsed_response["authkeywords"]["author-keyword"]["$"].titlecase.strip
          end
        end
        @bib << BibTeX::Entry.new(entry)
      else
        puts "Invalid record!"
        puts url
        puts "Continuing"
      end
        
    elsif response.timed_out?
      # aw hell no
      puts "got a time out"
    elsif response.code == 0
      # Could not get an http response, something's wrong.
      puts response.return_message
    else
      # Received a non-successful http response.
      puts "major fail"
      puts response.body
      puts "HTTP request failed: " + response.code.to_s
    end
  end
  request
end

def format_authors(authors)
  authors.map do |author|
    "#{author['ce:surname']}, #{author['ce:given-name']}"
  end.join(" and ")
end

def is_valid_record(data)
  data["dc:creator"] && BIBTEX_TYPES[data["subtypeDescription"]]
end

make_keyword_request(@search_string).run

#urls = File.open("artificial_creativity.txt").read
@urls.each_with_index do | url , i |
  puts "Querying #{i+1} of #{@urls.length}"
  make_abstract_request(url.strip).run
end

#make_abstract_request("https://api.elsevier.com/content/abstract/scopus_id/84900523833").run
#print @bib.to_s
File.write("#{@search_string.remove('"').squish}.bib", @bib.to_s)

9.2 GPT categorization prompt

A media is intended as the field of application addressed by the paper in question.
The theoretical scope of a paper is intended as the focus of the paper's research question.
The computational approach is intended as the broad category of algorithms and implementations discussed in the paper.
I need your help identifying the domain of this paper:

Title: {title}

Abstract: {abstract}

Journal: {journal}

Please assign it to one of the following media:

  - No medium
  - Visual, images and movies
  - Culinary recipes
  - Design, Urban Design, Architecture
  - Writing, narrative and language
  - Music and musical composition
  - Game design
  - Concepts
  - Multi-modal

Please also help me assign this paper to one of the following theoretical scopes based on the
research question that the paper is addressing:
  - Evaluation: the paper discusses the evaluation of creative artifacts
  - Theory: the paper discusses a particular theory or hypothesis about creativity
  - System: the paper presents a technical implementation of a specific system or algorithm developed by the authors
  - Other: the scope of the paper does not fit any of the other categories

Please also assign one of these computational approaches:
  - Rule based: these are non-data driven methods that follow deterministic rules and definitions, such as expert systems
  - Evolutionary algorithms: genetic algorithms and other evolutionary methods
  - Data driven: this includes Deep learning, machine learing and other methods that are based on datasets
  - Other: does not belong to any of the above or the approach is not clearly specified


Return the domain and the explanation of your choice in this format:
[ the media of the paper] | [ the theoretical scope ] | [the computational approach] | [ your explanation]

for example:

Multi-modal | Evaluation of creativity | Data driven | This paper discusses [...] so it belongs to [...]

Follow this format strictly and do not add any other words or prefix such as "Scope:" or "Medium:" before the domain and category you pick.

9.3 Top cited paper and their research question

The following table displays the 25 papers with most citations in the corpus. The number of citations was not considered in the systematic literature review, because of the possible bias it may introduce. All topics have been considered as a equally important regardless of whether the paper was cited or not. Citations also are depended on publication date, which may yield further bias in the analysis.