Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve TeSS scraping #4479

Merged
merged 10 commits into from
Nov 8, 2023
Merged

Improve TeSS scraping #4479

merged 10 commits into from
Nov 8, 2023

Conversation

hexylena
Copy link
Member

@hexylena hexylena commented Nov 2, 2023

  • adds version field
  • improved educational roles
  • separate out contributors
  • remove redundant "Hands on for "X"' which is just excessive
  • standardised draft status

shiltemann
shiltemann previously approved these changes Nov 2, 2023
[]
elsif material.key?('contributions')
material['contributions']
.reject{|k| k== 'funding'}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 [rubocop] <Layout/SpaceBeforeBlockBraces> reported by reviewdog 🐶
Space missing to the left of {.

Suggested change
.reject{|k| k== 'funding'}
.reject {|k| k== 'funding'}

[]
elsif material.key?('contributions')
material['contributions']
.reject{|k| k== 'funding'}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 [rubocop] <Layout/SpaceInsideBlockBraces> reported by reviewdog 🐶
Space between { and | missing.

Suggested change
.reject{|k| k== 'funding'}
.reject{ |k| k== 'funding'}

[]
elsif material.key?('contributions')
material['contributions']
.reject{|k| k== 'funding'}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 [rubocop] <Layout/SpaceAroundOperators> reported by reviewdog 🐶
Surrounding space missing for operator ==.

Suggested change
.reject{|k| k== 'funding'}
.reject{|k| k == 'funding'}

[]
elsif material.key?('contributions')
material['contributions']
.reject{|k| k== 'funding'}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 [rubocop] <Layout/SpaceInsideBlockBraces> reported by reviewdog 🐶
Space missing inside }.

Suggested change
.reject{|k| k== 'funding'}
.reject{|k| k== 'funding' }

elsif material.key?('contributions')
material['contributions']
.reject{|k| k== 'funding'}
.reject{|k| k== 'authorship'}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 [rubocop] <Layout/SpaceBeforeBlockBraces> reported by reviewdog 🐶
Space missing to the left of {.

Suggested change
.reject{|k| k== 'authorship'}
.reject {|k| k== 'authorship'}

Comment on lines 384 to 386
if material.key?('short_id')
data['identifier'] = "https://gxy.io/GTN:#{material['short_id']}"
end
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 [rubocop] <Style/IfUnlessModifier> reported by reviewdog 🐶
Favor modifier if usage when having a single-line body. Another good alternative is the usage of control flow &&/||.

Suggested change
if material.key?('short_id')
data['identifier'] = "https://gxy.io/GTN:#{material['short_id']}"
end
data['identifier'] = "https://gxy.io/GTN:#{material['short_id']}" if material.key?('short_id')

Comment on lines 402 to 403
data['name'] = material['title']
else
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 [rubocop] <Style/IdenticalConditionalBranches> reported by reviewdog 🐶
Move data['name'] = material['title'] out of the conditional.

Suggested change
data['name'] = material['title']
else
else

Comment on lines 405 to 406
data['name'] = material['title']
end
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 [rubocop] <Style/IdenticalConditionalBranches> reported by reviewdog 🐶
Move data['name'] = material['title'] out of the conditional.

Suggested change
data['name'] = material['title']
end
end

@@ -513,6 +547,13 @@ def to_jsonld(material, topic, site)
data['author'] = authors
end

# Add non-author contributors
if material.key?('contributions')
data['contributor'] = get_contributors(material).map { |x|
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 [rubocop] <Style/BlockDelimiters> reported by reviewdog 🐶
Avoid using {...} for multi-line blocks.

Suggested change
data['contributor'] = get_contributors(material).map { |x|
data['contributor'] = get_contributors(material).map do |x|

w['server'] = server
w
end
rescue
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 [rubocop] <Style/RescueStandardError> reported by reviewdog 🐶
Avoid rescuing without specifying an error class.

Suggested change
rescue
rescue StandardError

shiltemann
shiltemann previously approved these changes Nov 2, 2023
@shiltemann shiltemann merged commit b8cff07 into main Nov 8, 2023
2 checks passed
@bgruening bgruening deleted the coydog-banjo branch November 8, 2023 10:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants