Gfrktyl

I finished my first program in Ruby and I would like to share it with you so I can get some suggestions or recommendations. I'd really like to hear them because I am learning and I want to have a solid base. The program has a basic function: it counts the 15 more common words in a document. It is useful in literary studies because it allows the user to know which words is repeating more times, maybe without noticing it. This is how the program works:

The user is able to drag the document into the terminal and to explicitly write the path. Then, the script uses the gem 'yomu' to read it. It reads pdf and docx as well as other files that aren't important for me now. After that, the script makes an array of the words of the document. At this point, the script removes words like prepositions or connectors which aren't important to the user. Finally, it outputs the results in a table through a gem called 'hirb'.

The scripts also offers the possibility to modify the list of words that are going to be removed from the original words. For example, if a document has "Written by Rick Deckard" in every page as a footnote, the user is able to include the words "Written", "by", "Rick" and "Deckard" in the list and they aren't going to be counted. The user can modify the list and save the changes for that session or for ever.

So, there are three scripts: "wordsworth.rb", "core.rb", "forbidden_list_modify_eng.rb". The former script is in the main directory, while the other two are in the path 'languages/english'. That's because I adapted the program to spanish and catalan as well. Anyway, the main script is "wordsworth.rb", which asks the user their language and redirect him or her to the chosen language folder (english, in this case). Once in the english section, the script calls "core.rb", which is the main script of the english section. It means that this scripts is going to do the chief task of the program: count the words and output the results. However, "core.rb" offers the user to modify the list, and this option redirects the user to the third script "forbidden_list_modify_eng.rb" (sorry for the name).

Now I am going to post the three scripts and explain them in detail:

#needs to install 'yomu' and 'hirb' gems



require 'hirb'

require 'yomu'

require 'fileutils'



require_relative "languages/english/forbidden_list_modify_eng.rb"

require_relative "languages/english/core.rb"



def language

  puts "Choose your language:"

  puts "1. Català"

  puts "2. English"

  puts "3. Castellano"

  idiom = gets.chomp.to_s

  case idiom

  when "1", "català", "Català", "CATALÀ"

    system "clear" or system "cls"

    menu_principal_cat

  when "2", "english", "English", "ENGLISH"

    system "clear" or system "cls"

    main_menu

  when "3", "castellano", "Castellano", "CASTELLANO"

    system "clear" or system "cls"

    menu_principal_cast

  else

    system "clear" or system "cls"

    puts "You have to choose a language in the list."

    puts "-----------------------------------------------------"

    language

  end

end



language()

This script is pretty straight-forward, except for the last part, which I'll explain later. It first requires the needed gems and calls the needed scripts. After that, a menu asks the language to the user so it can redirect him or her to the correct language. We suppose the user choose 2 and gets redirected to the function "main_menu", which lands the user to the "core.rb" script:

def tutorial

  puts "The program works like that... (blah blah blah)"

end



def main_menu

  retrieve_forbidden_words = File.read('languages/english/english_list.txt')

  english_list = retrieve_forbidden_words.split("n")

  puts "Choose what you want to do by writing its number:"

  puts "1: Analyze a document."

  puts "2: Modify the list of forbidden words."

  puts "3: Help me to use this program."

  puts "4: Exit the program."

  main_choice = gets.chomp.to_s

  case main_choice

  when "1"

    system "clear" or system "cls"

    english(english_list)

  when "2"

    system "clear" or system "cls"

    modification_menu

  when "3"

    system "clear" or system "cls"

    tutorial

  when "4"

    exit

  else

    main_menu

  end

end



def english(received_list)

  puts "Drag in here the document you want to analyze:"

  provided_path = gets.chomp.to_s.tr("'", "").rstrip

  execution(provided_path, received_list)

end



def execution (provided_path, forbidden_list)

  data = File.read provided_path

  doc = Yomu.read :text, data



  text_listed = doc.to_s.downcase.gsub(/p{^Alnum}/, " ").split(" ")

  pruned_text = text_listed - forbidden_list



  frequency = Hash.new 0

  pruned_text.each { |word| frequency[word] += 1 }

  sorted_frequency = Hash[frequency.sort_by{ | word, times | -times }[0..20]]

  toptimes = sorted_frequency.values

  topwords = sorted_frequency.keys



  puts Hirb::Helpers::AutoTable.render(sorted_frequency, headers: {0 => 'Word', 1 => 'Times'}, description: false)

  puts "-----------------------------------------------------"

  main_menu



end

In the "main_menu" the user can choose to analyze the document. If so, the script asks the path of the document in the function "user_doc_path", which takes also the list of forbidden words in English from a .txt file which comes in the program by default. It is a simple file named "english_list.txt" which contains all the words to be removed from the document separated by a line break. So, the user provides the path of the document and the script opens it using 'yum' gem and sort it to get the words in an array. After that, the script creates a hash that stores the number of times each word is repeated throughout the text. Then, the program orders the hash, reverses it and get the 15 top results which finally output in a table provided by 'hirb' gem.

If the user choose to modify the list of words that are removed from the document, he or she is redirected to the "forbidden_list_eng_modify.rb":

def remove_words(list_to_remove)

  puts "Write the words you want to remove from the list. If there are more than one, divide them by commas."

  remove_propose = gets.chomp.tr(" ", "").to_s.split(",")

  new_list = list_to_remove - remove_propose



  File.new("languages/english/english_backup.txt", "w")

  File.open("languages/english/english_backup.txt", "w+") do |f|

  f.puts(list_to_remove)

  end



  File.open("languages/english/english_list.txt", "w+") do |f|

  f.puts(new_list)

  end



  puts "The words have been removed. Do you want to see the new list? [Yes/No]"

  showit = gets.chomp.tr(" ", "").to_s

  case showit

  when "Yes", "yes", "Y", "y"

    system "clear" or system "cls"

    print new_list

    puts "-----------------------------------------------------"

    modification_menu

  when "No", "no", "N", "n"

    system "clear" or system "cls"

    modification_menu

  end

end



def add_words(list_to_add)

  puts "Write the words you want to add to the list. If there are more than one, divide them by commas."

  add_propose = gets.chomp.tr(" ", "").to_s.split(",")

  new_list = list_to_add + add_propose



  File.new("languages/english/english_backup.txt", "w")

  File.open("languages/english/english_backup.txt", "w+") do |f|

  f.puts(list_to_add)

  end

  File.open("languages/english/english_list.txt", "w+") do |f|

  f.puts(new_list)

  end



  puts "The words have been added. Do you want to see the new list? [Yes/No]"

  showit = gets.chomp.tr(" ", "").to_s

  case showit

  when "Yes", "yes", "Y", "y"

    system "clear" or system "cls"

    print new_list

    puts "-----------------------------------------------------"

    modification_menu

  when "No", "no", "N", "n"

    puts "-----------------------------------------------------"

    modification_menu

  end

end



def modification_menu

retrieve_forbidden_words = File.read('languages/english/english_list.txt')

original_list = retrieve_forbidden_words.split("n")



puts "This list contains all the words that are automatically removed from the document you want to analyze. The words in it are widely-used words with no special meaning like connectors or prepositions. However, you can modify the list as you want."

puts "1: Show all the words of the list."

puts "2: Remove words."

puts "3: Add words."

puts "4: Save the new list for other sessions."

puts "5: Return to the main menu."

list_choice = gets.chomp.to_s

case list_choice

when "1"

  system "clear" or system "cls"

  print original_list

  puts ""

  puts "-----------------------------------------------------"

  modification_menu

when "2"

  system "clear" or system "cls"

  remove_words(original_list)

when "3"

  system "clear" or system "cls"

  add_words(original_list)

when "4"

  backup_existence = File.file?('languages/english/english_backup.txt')

  if backup_existence == TRUE

    File.delete('languages/english/english_backup.txt')

    system "clear" or system "cls"

    puts "Changes has been succesfully saved."

    puts "-----------------------------------------------------"

    modification_menu

  end

  if backup_existence == FALSE

    system "clear" or system "cls"

    puts "The list hasn't been modified, so it's nothing to save."

    puts "-----------------------------------------------------"

    modification_menu

  end

when "5"

  system "clear" or system "cls"

  main_menu

else

  system "clear" or system "cls"

  modification_menu

end

end

This script shows the user a menu which offers the possibility of showing the list, adding or removing some words and saving the changes for ever. The first option, show the complete list, just prints the output of the "english_list.txt", which is the file where the words are stored. The "add_words", as the "remove_words" function, asks the user to write the words that want to get removed separating them by commas. The script just removes the provided array from the default list. At this point, the script creates a "english_backup.txt" which is the back up in case the user closes the program without saving. The modified list is named "english_list.txt", and it works as the new default list, so the user can analyze more texts in the same session with that list. If the user chooses to save the changes, the "english_backup.txt" is destroyed and the new "english_list.txt" is the main list. If the user exits, the "english_backup.txt" will get its former name ("english_list.txt") and the unsaved list file will be removed. This is achieved in the former script, "wordsworth.rb", when END is detected it proceeds to remove the unsaved file and restore the backup as main list.

This is a really interesting project, and looks pretty solid. I'm just
going to go through and give you some ruby style tips as I see them.
I've annotated all of the files with my thoughts.

First of all, you use lists in a couple places. Because of that, I'd
suggest that you create a generic list method right above your #language
method:

def list(*options, retry_msg: '')

  options = Array(options).flatten.map(&:to_s).uniq

  invalid_opts = (0..options.length).map(&:to_s)

  throw "You cannot pass a number as an option" if options.any? { |opt| invalid_opts.include?(opt) }

  loop do

    puts options.map.with_index { |opt, i| "#{i}. #{opt}" }

    selection = gets.chomp

    system "clear" or system "cls"

    if (0..options.length-1).map(&:to_s).include? selection

      return options[selection.to_i]

    else

      selection = options.select { |opt| opt.downcase == selection.downcase }.first

      return selection unless selection.nil?

    end

    puts retry_msg

  end

end

Side note: Since this would remove your lists throughout, I just wanted
to add some code style tips in case you decided to leave the list in.
Here are my comments on the list in your #language method:

def language

  # I would simplify the listing of languages (and make it more extensible)

  # by having an array of languages you support.

  puts "Choose your language:"

  # I've created the array using the %w literal, which creates an array

  # of words split on the spaces.

  languages = %w[Català English Castellano]

  # puts "1. Català"

  # puts "2. English"

  # puts "3. Castellano"

  # And then print them out: (puts automatically adds newlines

  # between the array elements)

  puts languages.map.with_index { |lang, index| "#{index}. #{lang}" } 

  idiom = gets.chomp.to_s

  # Here, you can convert the idiom to lowercase so you don't need

  # to have different capitalizations in your when statements

  case idiom.downcase

  when "1", "català"#, "Català", "CATALÀ"

    system "clear" or system "cls"

    menu_principal_cat

  when "2", "english"#, "English", "ENGLISH"

    system "clear" or system "cls"

    main_menu

  when "3", "castellano"#, "Castellano", "CASTELLANO"

    system "clear" or system "cls"

    menu_principal_cast

  else

    system "clear" or system "cls"

    puts "You have to choose a language in the list."

    puts '-'*53 # You can use multiplication instead of typing the char so many times

    language

  end

end

Anyways, here's an actual review of your main file:

require 'hirb'

require 'yomu'

require 'fileutils'



require_relative "languages/english/forbidden_list_modify_eng.rb"

require_relative "languages/english/core.rb"



# The addition of the list method means that all this code doesn't need

# to be wrapped in a language method

# def language

  puts "Choose your language:"

  # puts "1. Català"

  # puts "2. English"

  # puts "3. Castellano"

  # idiom = gets.chomp.to_s

  retry_msg  = "You have to choose a language in the list.n"

  retry_msg += '-'*53

  case list(%w[Català English Castellano], retry_msg: retry_msg)

  when "Català"

    menu_principal_cat

  when "English"

    main_menu

  when "Castellano"

    menu_principal_cast

  end

# end



# language()

And, for the second file:

def tutorial

  # If this is really long, it might be a good idea to put

  # the contents of the tutorial in a text file and instead

  # use puts File.read("path/to/file")

  puts "The program works like that... (blah blah blah)"

end



def main_menu

  # I've renamed the variable to be forbidden_words (from retrieve_forbidden_words) because the variable is the forbidden words, not the action of retrieving them.

  forbidden_words = File.read('languages/english/english_list.txt')

  english_list = forbidden_words.split("n")

  # Here we can use the magic list method

  puts "Choose what you want to do by writing its number:"

  # puts "1: Analyze a document."

  # puts "2: Modify the list of forbidden words."

  # puts "3: Help me to use this program."

  # puts "4: Exit the program."

  # main_choice = gets.chomp.to_s

  # case main_choice

  options = 

  options << 'Analyze a document.'

  options << 'Modify the list of forbidden words.'

  options << 'Help me to use this program.'

  options << 'Exit the program.'

  case list(options)

  when 'Analyze a document.'

    english(english_list)

  when 'Modify the list of forbidden words.'

    modification_menu

  when 'Help me to use this program.'

    tutorial

  when 'Exit the program.'

    exit

  end

end



def english(received_list)

  puts "Drag in here the document you want to analyze:"

  provided_path = gets.chomp.to_s.tr("'", "").rstrip

  execution(provided_path, received_list)

end



def execution (provided_path, forbidden_list)

  data = File.read provided_path

  doc = Yomu.read :text, data



  text_listed = doc.to_s.downcase.gsub(/p{^Alnum}/, " ").split(" ")

  pruned_text = text_listed - forbidden_list



  frequency = Hash.new 0

  pruned_text.each { |word| frequency[word] += 1 }

  sorted_frequency = Hash[frequency.sort_by{ | word, times | -times }[0..20]]

  toptimes = sorted_frequency.values

  topwords = sorted_frequency.keys



  puts Hirb::Helpers::AutoTable.render(sorted_frequency, headers: {0 => 'Word', 1 => 'Times'}, description: false)

  puts "-----------------------------------------------------"

  main_menu



end

And, the final file:

def remove_words(list_to_remove)

  puts "Write the words you want to remove from the list. If there are more than one, divide them by commas."

  remove_propose = gets.chomp.tr(" ", "").to_s.split(",")

  new_list = list_to_remove - remove_propose



  File.new("languages/english/english_backup.txt", "w")

  File.open("languages/english/english_backup.txt", "w+") do |f|

  f.puts(list_to_remove)

  end



  File.open("languages/english/english_list.txt", "w+") do |f|

  f.puts(new_list)

  end



  puts "The words have been removed. Do you want to see the new list? [Yes/No]"

  showit = gets.chomp.tr(" ", "").to_s

  case showit

  when "Yes", "yes", "Y", "y"

    system "clear" or system "cls"

    print new_list

    puts "-----------------------------------------------------"

    modification_menu

  when "No", "no", "N", "n"

    system "clear" or system "cls"

    modification_menu

  end

end



def add_words(list_to_add)

  puts "Write the words you want to add to the list. If there are more than one, divide them by commas."

  add_propose = gets.chomp.tr(" ", "").to_s.split(",")

  new_list = list_to_add + add_propose



  File.new("languages/english/english_backup.txt", "w")

  File.open("languages/english/english_backup.txt", "w+") do |f|

  f.puts(list_to_add)

  end

  File.open("languages/english/english_list.txt", "w+") do |f|

  f.puts(new_list)

  end



  puts "The words have been added. Do you want to see the new list? [Yes/No]"

  showit = gets.chomp.tr(" ", "").to_s

  case showit

  when "Yes", "yes", "Y", "y"

    system "clear" or system "cls"

    print new_list

    puts "-----------------------------------------------------"

    modification_menu

  when "No", "no", "N", "n"

    puts "-----------------------------------------------------"

    modification_menu

  end

end



def modification_menu

retrieve_forbidden_words = File.read('languages/english/english_list.txt')

original_list = retrieve_forbidden_words.split("n")



puts "This list contains all the words that are automatically removed from the document you want to analyze. The words in it are widely-used words with no special meaning like connectors or prepositions. However, you can modify the list as you want."

puts "1: Show all the words of the list."

puts "2: Remove words."

puts "3: Add words."

puts "4: Save the new list for other sessions."

puts "5: Return to the main menu."

list_choice = gets.chomp.to_s

case list_choice

when "1"

  system "clear" or system "cls"

  print original_list

  puts ""

  puts "-----------------------------------------------------"

  modification_menu

when "2"

  system "clear" or system "cls"

  remove_words(original_list)

when "3"

  system "clear" or system "cls"

  add_words(original_list)

when "4"

  backup_existence = File.file?('languages/english/english_backup.txt')

  if backup_existence == TRUE

    File.delete('languages/english/english_backup.txt')

    system "clear" or system "cls"

    puts "Changes has been succesfully saved."

    puts "-----------------------------------------------------"

    modification_menu

  end

  if backup_existence == FALSE

    system "clear" or system "cls"

    puts "The list hasn't been modified, so it's nothing to save."

    puts "-----------------------------------------------------"

    modification_menu

  end

when "5"

  system "clear" or system "cls"

  main_menu

else

  system "clear" or system "cls"

  modification_menu

end

end

Side note: My apologies for taking so long to get back to this, I just got busy IRL and couldn't finish.