1. Teams
1.1. Spectre (Henley) and Procodia (Poland) worked well together
1.2. Numbers dwindling in UK are not replaced, meaning that we have one less team in next release.
1.3. Enigma and Front-End teams worked well together. Ron really helped us when he joined.
1.4. New feature development would have been easier for Enigma if we could also make Front-End changes.
1.5. Good cooperation between Procodia and Spectre.
1.6. Procodia was always prepared to demo however some demo sessions was cancelled.
1.6.1. Stephen to stop cancelling the demo when there is less than 10 minutes of content.
1.6.2. Teams should always show work on completed stories even if the demo is just a set of points on a slide.
1.6.2.1. In the same way the demo should remain relevant
1.6.2.2. Teams dont need to prove that they have done the work with long demos. Talking to a single slide is ok.
1.7. Integration between vanorama and specodia very smooth
2. Hardening
2.1. Hardening Test cycles needs better management/ownership/direction. There needs to be more investment in improving the tests and removing duplication. In some cases there are 4 tests that can be combined into 1.
2.1.1. Test case maint is an ongoing requirement for everyone
2.1.2. Formal test plan to be created - already being created by QA folks
2.2. Was difficult to QA new features until both back-end and front-end stories were complete.
2.3. Too reliant on manual test cases. Need to automate manual test cases to reduce what's left to do in Hardening.
2.4. We haven’t got well defined what template can go with which AC and delivery
2.5. QA team members did less test than expected as we had a lot of questions on test cases
2.6. Digital Delivery Integration issues held up hardening testing
2.6.1. Need to focus on DD and EOV integration testing in H release
2.6.1.1. Enigma team are looking to integrate with EOV early and often in H release
3. Backlog Grooming
3.1. Hornet Release
3.1.1. Time needs to be planned for getting next release backlog ready
3.1.2. Time needs to be allocated for spiking for I release before H ends
3.2. Galaxy Release
3.2.1. Grooming the backlog every sprint help get us ready for sprint planning
4. AWS
4.1. Excellent set of tools, well-used
4.2. Still have a mindset of holding onto VMs, resulting in unnecessary costs. Need to be more conscious of costs and delete what's not actively in use
4.2.1. Do we need more training on the use of AWS?
4.2.1.1. Terry / Bill working on training
4.3. Infrastructure made it easy to test installation of NDA
4.4. Use of AWS has made CICD reliable and valuable - no longer have various systems (UrbanCode + CloudForms) tied together in a fragile way
4.4.1. We have created a best practice system that we are starting to educate the rest of the business about.
5. QA
5.1. Quality Owners were a help
5.2. Functional/Service strategy worked well
5.3. Test case maintenance an ongoing requirement
5.4. When CICD fails teams should be able to manually install the build so that testing isn't blocked
5.4.1. If they are issues with deployments on new builds on Team City, we can build deployment choosing build and scripts version – we have it documented on confluence.
5.5. Automation needs more attention
5.5.1. All teams have objective to do automation in H
5.6. Lost a number of QA resource during Galaxy. QA capacity further reduced when QAs are also Scrum Masters which accounts for 20-25% of their time.
5.7. Move to a new Vault instance in hardening was not picked up by all QA. Kept sending jobs to old instance and could not see them in EO Archive displays
5.8. iBeta contractors / Rebecca performed a massive amount of testing
5.8.1. Develop a plan to do continuous regression testing
6. Documentation
6.1. Process improved - Installation guide on Confluence useful
6.2. Having one place on confluence for all installation documentation was very good idea.
6.2.1. Need to build on the processes that were adopted in H to put into Confluence as our knowledge base.
6.2.2. Scrum masters need to work to make sure that we record decisions and relevant notes in Confluence.
6.2.3. Stephen to ask GES to speed up confluence
6.2.4. Stephen: Alignment call / SOS: New pages in confluence.
7. Performance Analysis
7.1. We're getting numbers / tools under our belt
7.2. ACM performance testing was pushed into hardening because of other priorities. This was risky, but lucking no changes were required.
7.2.1. Performance testing needs to be baked into the original estimation.
8. Digital Delivery (eMessaging)
8.1. Quite a lot of confusion to resolve integration
8.1.1. Waited too long to start integrating
8.1.1.1. Need to integrate early and make sure that we have good documentation.
8.1.1.2. Need to automate the deployment and integration
8.1.1.2.1. DD team is starting to work in AWS - this should help
8.2. Integration documentation was out of date and incomplete.
8.3. It wasn’t clear which team should be responsible for which integration part (for example on vendors).
9. Vault
9.1. Job delivery to shared Vault instance is done using RDP with cut and pasted to the Vault hot folder. This is very prone to interruption by someone else logging into the shared Vault system.
10. NDA
10.1. Dan Sullivan led NDA very well, made it easy for the teams to focus on what was required and track progress
10.2. Work was well-coordinated across multiple teams
10.3. Incremental approach to moving components into NDA worked well. Led to early merge of NDA branch being possible.
10.4. Developer install and test process much improved
10.5. We have EO installation process documented on confluence so if there are problems with deployments on CI/CD anyone can install it manually
10.6. Dan divided NDA work to consumable parts and he was always answering questions very quickly.
10.6.1. Good example of how having good backlogs / defined sprints can make a project go well
11. Release
11.1. Planning
11.1.1. Potential release content too little understood at time of release planning
11.1.2. Too many people involved
11.1.2.1. Making sure that no matter the number of people involved there is a single decision maker.
11.1.2.2. Clear roles needed in release ceremonies
11.1.3. Preparation for the next release started in Galaxy
11.1.3.1. Product backlog grooming sessions needed through the release
11.2. Stand-ups
11.2.1. Get the right people involved in triage, I'm not sure the scrum masters have the correct knowledge or are able to make the correct decisions.
11.2.2. Not enough focus on customer and usability when it comes to bugs raised at the end of the release.
11.2.2.1. Customer impact is not given high enough value in the triage process.
11.2.2.2. PdM are working to make sure that we dont focus on single customers over the needs of the market.
11.2.2.3. PdM are working to make sure that they are talking to more customers. Looking to restart user groups sessions.
11.2.3. Stand-ups could be more effective if attendees came to the meeting more prepared
11.3. Creating release deliverables
11.3.1. Still a multiple day effort
11.3.1.1. Bill Horst is looking at the process and to make sure it is a single day process. That could be via docs or changes,
11.3.2. Release artifacts in Nexus were cached. Required a reset
12. PdM
12.1. Need to manage expectations in the field. When ACM phase 1 was delivered it got negative feedback because the implementation did not go far enough.
12.1.1. Pdm to make sure that all feedback - good and bad are passed back into the team
12.1.2. Stephen to add section to alignment call on customer feedback on the previous release features
12.2. Need to attend sprint demos more and provide feedback on features as they are being developed
12.2.1. Product to work on comms to enablement group.
12.2.2. Product to work to give more feedback in demos and to ensure that the team attends the demo.
12.2.3. Terry to start looking to have customer feedback into the sprints.
12.2.4. Stephen to reinforce to the demo attendees to give feedback
13. UX
13.1. UX needs to be ready prior to development starting on stories. The correct level of detail required needs to be agreed
13.1.1. Ownership TBD
13.1.2. UX is interviewing customers at the moment - updates to follow.
14. Sprints
14.1. There needs to be more involvement from PdM in sprint deliverables. Use the software at the end of the sprint to get agreement that what has been produced matches the requirements
14.1.1. More involvement in the demo
14.1.2. Direct customer input into the demo
14.1.2.1. Terry is investigating a way to demo to customers that does not stifle conversation in a demo.
14.1.2.2. The detail in the team demo may not be consumable for customers. Ongoing monthly enablement that can be recorded for customers.
14.1.3. More PdM involvement in the validation of the release deliverables
14.1.4. Action: Scrum masters need to be open enough to ask PdM when they feel that something is missing.
14.1.4.1. Team can make sure they close the loop on the requirements with the product owner. The demo should not be the first time that the PO sees the software.
14.1.4.2. One suggestion is to have a feature demo session for the PO as things are completed before the end of the sprint.
14.2. Enigma managed to balance release stories with Portrait support commitments.
14.2.1. In the short term the team will continue this pattern
14.3. Phoenix team achieved Galaxy commitments alongside assisting the maintenance team with escalated items
14.3.1. Keep working as a team to balance our commitments
14.3.2. We need to stop putting ourselves in the position where we need to make heroic efforts
14.4. Retros
14.4.1. Teams don't seem to have the freedom to fully go into details about what's not working. This maybe because once it's repeated a few times and nothing happens, there is no point in discussing it further. Teams don't want to blame others.
14.4.1.1. ROR: Not everyone is completely honest in ROR
14.4.1.2. We need to keep bringing the issues up. - Yaakov to try to be more responsive
14.4.1.3. Technical road bumps have been slow to be tackled - eg the merging of the source code trees in SVN
14.4.1.3.1. Bill Horst should start to work through CI/CI items
14.4.1.3.2. Tech Debt has been included in the H planning
14.4.1.4. Teams don't feel empowered to work on some issues
14.4.1.4.1. Need to help empower teams to take things on
14.4.1.4.2. Mgmt team: Need to discuss action items
14.4.1.5. Scrum Masters: Dont continue to suffer in silence.
14.4.1.5.1. Speak to your line manager
14.4.1.5.2. Email Yaakov with email titled "you are not listening"
14.4.2. Ensure scrum masters feed back information for other team retros to enable learning
14.4.2.1. Scrum masters need to feed information up and down
14.4.2.1.1. Not just forwarding on meeting notes
14.4.2.1.2. SM role is to keep the team informed as well as to communicate information from the team outwards
14.4.2.1.3. Do we need to go back to bi-monthly development team meetings?
15. Launch enablement
15.1. Did a better job of preparing Services and Pre-sales for receipt of the release
15.1.1. Good job done by the team.
15.1.2. Roger Pilc has been talking about the great job done by team
15.1.3. Stephen: Build 4.4 enablement plan early.
15.1.4. Good work from PdM and Shirley
15.1.5. Recordings from mid release demos to build the enablement
16. Key
16.1. Start Doing
16.1.1. Action / Owner: ?