Refactor feature-combining layers in DIET, TED, and ResponseSelector (R…

…asaHQ#7589)

* Port all refactoring changes from branch e2e-feature-combining-layers into 2.2.x-based branch.

* Add e2ebot with forms, entities and slots.

* Reformat code.

* Remove unused code and move new layers into rasa_layers.py.

* Make small changes based on review comments.

* Fix Keras errors in DIET.

* Hard-code training of DIET and TED to use eager mode (to see which tests fail).

* Add step to clear poetry cache

* Clear poetry cache before a virtual env is created

* Don't use force for rm - windows

* Refactor rasa_layers to consume all inputs in single  arg.

* Disable eager training of TED.

* Fix a trivial mistake in DIET.

* Check differences in results between refactored DIET and the master version.

* Polish up and document the ConcatenateSparseDenseFeatures class.

* Fix a fatal typo.

* Document the ConcatenateSequenceSentenceFeatures custom layer.

* Fix typo after refactoring new layers.

* Continue adding docstrings, almost finish RasaInputLater.

* Comment in RasaSequenceLayer and DIET on bits I previously didn't fully grasp.

* Finish documenting all code in rasa_layers.

* Clean up & document code in refactored parts of TED & DIET.

* Remove unnecessary use of a kwargs dict for just 1 argument.

* Remove e2ebot1.

* Make small changes based on review comments.

* Make changes based on 2nd round of review comments.

* Change how masks are named; simplify the SeqSent layer.

* Remove None returns from custom layers.

* Fix a test to expect empty numpy array of attention weights when the DIET transformer has no layers.

* Make changes based on 3rd round of review comments.

* Move all mask computation from DIET and TED into custom layers (docstrings and comments to be adjusted later).

* Make changes based on 4th round of review comments.

* Move sequence/sentence feature combining logic into RasaFeatureCombiningLayer.

* Document some (but not all) changes in rasa_layers.py.

* Finish polishing up and commenting the code in rasa_layers.

* Clean up code in rasa_layers that checks that necessary features are present.

* Small cleanup and renaming in DIET.

* Fix typos in code.

* Make changes based on another review. Fix bug in masked LM.

* Update rasa/utils/tensorflow/rasa_layers.py

Co-authored-by: Daksh Varshneya <[email protected]>

* Update rasa/utils/tensorflow/rasa_layers.py

Co-authored-by: Daksh Varshneya <[email protected]>

* Update rasa/utils/tensorflow/rasa_layers.py

Co-authored-by: Daksh Varshneya <[email protected]>

* Update rasa/utils/tensorflow/rasa_layers.py

Co-authored-by: Daksh Varshneya <[email protected]>

* Update rasa/utils/tensorflow/rasa_layers.py

Co-authored-by: Daksh Varshneya <[email protected]>

* Update rasa/utils/tensorflow/rasa_layers.py

Co-authored-by: Daksh Varshneya <[email protected]>

* React to some more review comments.

* Tiny fix in docstrings.

* Draft changes to ResponseSelector.

* Factor out some shared functions.

* Add changelog for the masked language modeling bug fix.

* Tidy up some unnecessary expanding and squeezing of seq. length tensors in TED.

* Clean up last bits in ResponseSelector, make small fixes in rasa_layers.

* Start writing unit tests for rasa_layers.

* Add tests for exceptions and output shapes of Rasa layers.

* Fix broken import.

* Catch a bug in RasaSequenceLayer & add sensible checking to the sparse-dense concat layer.

* Add further tests for the new rasa layers.

* Finish drafting test cases for the new layers.

* Add test case for RasaSequenceLayer outputting made up token ids.

* Test new model layers both in training and testing mode.

* Change empty attention weights to None and adjust tests accordingly.

* Check that RasaSequenceLayer doesn't apply MLM at test time.

* Remove (probably) unnecessary TensorFlow flag.

* Final pass through the docstrings in new Rasa layers.

* Convert empty list to None everywhere in predictions[DIAGNOSTIC_DATA].

* Shorten long lines.

* Add a changelog entry.

* Add type annotations to new unit tests.

* Type-annotate forgotten function.

* Make small changes (typing & docstrings) based on review comments.

* Make small improvements based on review comments.

* Increase DIET's # of epochs for stable results of .

* Remove unused imports.

* Split a method into two to avoid conflicting method name vs behaviour.

* Fix a tiny bug.

* Add dummy docstrings to satisfy code quality checks.

* Add missing return type annotation.

Co-authored-by: tczekajlo <[email protected]>
Co-authored-by: Daksh Varshneya <[email protected]>

changelog/7589.bugfix.md

@@ -0,0 +1,3 @@
+Fix masked language modeling in DIET to only apply masking to token-level
+(sequence-level) features. Previously, masking was applied to both token-level and
+sentence-level features.

changelog/7589.improvement.md

@@ -0,0 +1,9 @@
+Improved the machine learning models' codebase by factoring out shared feature-processing
+logic into three custom layer classes:
+- `ConcatenateSparseDenseFeatures` combines multiple sparse and dense feature tensors
+into one.
+- `RasaFeatureCombiningLayer` additionally combines sequence-level and sentence-level
+features.
+- `RasaSequenceLayer` is used for attributes with sequence-level features; it
+additionally embeds the combined features with a transformer and facilitates masked
+language modeling.

data/test_response_selector_bot/config.yml

@@ -5,7 +5,7 @@ pipeline:
   - name: "CountVectorsFeaturizer"
   - name: "DIETClassifier"
     entity_recognition: False
-    epochs: 5
+    epochs: 8
     random_seed: 42
   - name: ResponseSelector
     epochs: 5