Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Or 2472 restructure #1014

Merged
merged 40 commits into from
Dec 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
c10de4a
test: or-2472 add first test for adres merger
emalfroy Nov 28, 2024
1a2e4a1
feat: or-2472 implement AddressMergerHandler
emalfroy Nov 28, 2024
41807b9
test: or-2472 add test for locations linked to multiple vcodes
emalfroy Nov 28, 2024
2acf25d
feat: or-2472 introduce TeHeradresserenLocatiesFinder
emalfroy Nov 28, 2024
8722c3e
feat: or-2472 complete implementation of TeHeradresserenLocatiesFinder
Nov 28, 2024
022da0b
feat: or-2472 add registration for ITeHeradresserenLocatiesFinder
emalfroy Nov 28, 2024
99bafe7
feat: or-2472 consume extra events for fusies
emalfroy Nov 28, 2024
71436c7
feat: or-2472 return LocatieLookupData instead of the LocatieLookupDo…
emalfroy Nov 29, 2024
554d46b
refactor: or-2472 use adres id inside merger handler instead of the c…
Nov 28, 2024
040366c
refactor: or-2472 rename folder grarsync to grarconsumer
emalfroy Nov 29, 2024
fa34714
refactor: or-2472 restructer grar consumer into folders
emalfroy Nov 29, 2024
e4b92fb
fix: or-2472 return IQueryAble for locatie lookup
emalfroy Nov 29, 2024
7828f75
fix: or-2472 group with destination adres idÃ
emalfroy Nov 29, 2024
fccd751
test: or-2472 add tests for LocatiesVolgensVCodeGrouper
emalfroy Nov 29, 2024
ee015e7
refactor: or-2472 rename folder
koenmetsu Nov 29, 2024
b8eeb21
test: or-2472 improve readability
koenmetsu Nov 29, 2024
fb00fba
refactor: or-2472 remove unused property
koenmetsu Nov 29, 2024
61dad4e
feat: or-2472 use correct handler depending on destination adres
koenmetsu Nov 29, 2024
265e913
feat: or-2472 introduce TeOntkoppelenAdresHandler
emalfroy Nov 29, 2024
f0aa08f
wip: or-2472 group inside LocatieFinder
koenmetsu Nov 29, 2024
248907a
wip: or-2472 use LocatieIdsPerVCodeCollection instead of dictionary
emalfroy Dec 2, 2024
2329a6e
refactor: or-2472 move LocatieFinder concepts to domain
koenmetsu Dec 2, 2024
6c784e5
test: or-2472 test LocatieFinder
koenmetsu Dec 2, 2024
cebd6fe
refactor: or-2472 improve clarity with folders
koenmetsu Dec 2, 2024
96be41e
test: or-2472 add test for LocatieIdsPerVCodeCollection map
emalfroy Dec 2, 2024
9bf2980
refactor: or-2472 move test into FusieEvents folder
emalfroy Dec 2, 2024
a305df0
feat: or-2472 implement TeOnkoppelenLocatieHandler
emalfroy Dec 2, 2024
fcdf1c6
test: or-2472 move Given_LocatieLookupDocuments into one class
emalfroy Dec 2, 2024
0384900
feat: or-2472 implement TeOntkoppelenLocatieHandler
emalfroy Dec 2, 2024
ab0cc49
refactor: or-2472 move more logic to domain
koenmetsu Dec 2, 2024
7891dc2
refactor: or-2472 separate commands and handlers, move to actions
koenmetsu Dec 2, 2024
9bec79a
refactor: or-2472 rename
koenmetsu Dec 2, 2024
dc53314
wip: or-2472 almost done refactoring, I promise
koenmetsu Dec 2, 2024
a915dbb
test: or-2472 fix test ÂÃÃ& cleanup files
emalfroy Dec 2, 2024
81205e0
fix: or-2472 fix missing registrations, add handler for sqs
koenmetsu Dec 3, 2024
985a2eb
feat: or-2472 use wrapper event to allow dynamic routing on same queue
koenmetsu Dec 3, 2024
5142f98
chore: or-2472 remove comment
koenmetsu Dec 3, 2024
effce41
chore: or-2472 revert localstack queues to eu-west-1
koenmetsu Dec 3, 2024
25af0ec
feat: or-2472 implement idempotencykey to heradresseerMessage for fusies
emalfroy Dec 3, 2024
efdcf95
test: or-2472 fix flaky test ontkoppel locaties
emalfroy Dec 3, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .localstack/init/ready.d/01_sqs-init.redrive.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
{
"ContentBasedDeduplication": "true",
"DeduplicationScope": "messageGroup",
"RedrivePolicy": "{\"maxReceiveCount\": \"3\", \"deadLetterTargetArn\": \"arn:aws:sqs:eu-west-1:000000000000:verenigingsregister-addresssmatch-dlq\"}"
}
2 changes: 0 additions & 2 deletions .localstack/init/ready.d/02_sqs-init.redrive.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
{
"ContentBasedDeduplication": "true",
"DeduplicationScope": "messageGroup",
"RedrivePolicy": "{\"maxReceiveCount\": \"3\", \"deadLetterTargetArn\": \"arn:aws:sqs:eu-west-1:000000000000:verenigingsregister-grarsync-dlq\"}"
}
2 changes: 1 addition & 1 deletion paket.lock
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,7 @@ NUGET
Autofac.Extensions.DependencyInjection (>= 9.0)
Be.Vlaanderen.Basisregisters.EventHandling (5.0)
Be.Vlaanderen.Basisregisters.Generators.Guid.Deterministic (4.0)
Be.Vlaanderen.Basisregisters.GrAr.Contracts (21.7)
Be.Vlaanderen.Basisregisters.GrAr.Contracts (21.15)
Be.Vlaanderen.Basisregisters.Middleware.AddProblemJsonHeader (3.0)
Be.Vlaanderen.Basisregisters.ProblemDetails (9.0)
System.Reflection.Metadata (>= 8.0)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@
</ItemGroup>

<ItemGroup>
<Folder Include="GrarConsumer\Handlers\Fusies\" />
<Folder Include="Infrastructure\ConfigurationBindings\" />
<Folder Include="Verenigingen\Contactgegevens\FeitelijkeVereniging\" />
<Folder Include="Verenigingen\Vertegenwoordigers\FeitelijkeVereniging\" />
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
namespace AssociationRegistry.Admin.Api.GrarConsumer.Finders;

using Grar.GrarUpdates.LocatieFinder;
using Schema.Detail;
using Marten;

public class LocatieFinder : ILocatieFinder
{
private readonly IDocumentStore _documentStore;

public LocatieFinder(IDocumentStore documentStore)
{
_documentStore = documentStore;
}

public async Task<LocatiesPerVCodeCollection> FindLocaties(params int[] adresIds)
{
return await FindLocaties(
adresIds
.Select(x => x.ToString())
.ToArray());
}

public async Task<LocatiesPerVCodeCollection> FindLocaties(params string[] adresIds)
{
var locatieIdsGroupedByVCode =
GroupByVCode(
await FindLocatieLookupDocuments(adresIds)
);

return LocatiesPerVCodeCollection.FromLocatiesPerVCode(locatieIdsGroupedByVCode);
}

public async Task<IEnumerable<LocatieLookupData>> FindLocatieLookupDocuments(string[] adresIds)
{
await using var session = _documentStore.LightweightSession();

return session.Query<LocatieLookupDocument>()
.Where(x => adresIds.Contains(x.AdresId))
.Select(x => new LocatieLookupData(x.LocatieId, x.AdresId, x.VCode));
}

private static Dictionary<string, LocatieLookupData[]> GroupByVCode(IEnumerable<LocatieLookupData> locaties)
{
return locaties
.GroupBy(x => x.VCode)
.ToDictionary(grouping => grouping.Key,
documents => documents.Select(x => x)
.ToArray());
}
}

Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
namespace AssociationRegistry.Admin.Api.GrarConsumer.Finders;

public record LocatieMetVCode(string VCode, int LocatieId);
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
namespace AssociationRegistry.Admin.Api.GrarSync;
namespace AssociationRegistry.Admin.Api.GrarConsumer.Kafka;

using Confluent.Kafka;
using Hosts.Configuration;
using Confluent.Kafka;

public class AddressKafkaConfiguration : ConsumerConfig
{
Expand Down
Original file line number Diff line number Diff line change
@@ -1,36 +1,39 @@
namespace AssociationRegistry.Admin.Api.GrarSync;
namespace AssociationRegistry.Admin.Api.GrarConsumer.Kafka;

using Infrastructure.AWS;
using Infrastructure.Extensions;
using Notifications.Messages;
using AssociationRegistry.Notifications;
using Be.Vlaanderen.Basisregisters.GrAr.Contracts.AddressRegistry;
using Confluent.Kafka;
using Grar.GrarUpdates.Fusies;
using Grar.GrarUpdates.Hernummering;
using IdentityModel;
using Infrastructure.AWS;
using Infrastructure.Extensions;
using Marten;
using Notifications.Messages;
using Polly;
using StraatHernummering;
using System.Text;

public class AddressKafkaConsumer : BackgroundService
{
private readonly AddressKafkaConfiguration _configuration;
private readonly TeHeradresserenLocatiesMapper _teHeradresserenLocatiesMapper;
private readonly SqsClientWrapper _sqsClientWrapper;
private readonly IFusieEventProcessor _fusieEventProcessor;
private readonly ITeHernummerenStraatEventProcessor _teHernummerenStraatEventProcessor;
private readonly IDocumentStore _store;
private readonly INotifier _notifier;
private readonly ILogger<AddressKafkaConsumer> _logger;

public AddressKafkaConsumer(
AddressKafkaConfiguration configuration,
TeHeradresserenLocatiesMapper teHeradresserenLocatiesMapper,
SqsClientWrapper sqsClientWrapper,
IFusieEventProcessor fusieEventProcessor,
ITeHernummerenStraatEventProcessor teHernummerenStraatEventProcessor,
IDocumentStore store,
INotifier notifier,
ILogger<AddressKafkaConsumer> logger)
{
_configuration = configuration;
_teHeradresserenLocatiesMapper = teHeradresserenLocatiesMapper;
_sqsClientWrapper = sqsClientWrapper;
_fusieEventProcessor = fusieEventProcessor;
_teHernummerenStraatEventProcessor = teHernummerenStraatEventProcessor;
_store = store;
_notifier = notifier;
_logger = logger;
Expand Down Expand Up @@ -80,17 +83,25 @@ private async Task Consume(CancellationToken stoppingToken)

switch (message)
{
case AddressWasRetiredBecauseOfMunicipalityMerger addressWasRetiredBecauseOfMunicipalityMerger:
await _fusieEventProcessor.Process(
addressWasRetiredBecauseOfMunicipalityMerger.AddressPersistentLocalId,
addressWasRetiredBecauseOfMunicipalityMerger.NewAddressPersistentLocalId,
idempotenceKey);
break;

case AddressWasRejectedBecauseOfMunicipalityMerger addressWasRejectedBecauseOfMunicipalityMerger:
await _fusieEventProcessor.Process(
addressWasRejectedBecauseOfMunicipalityMerger.AddressPersistentLocalId,
addressWasRejectedBecauseOfMunicipalityMerger.NewAddressPersistentLocalId,
idempotenceKey);
break;

case StreetNameWasReaddressed streetNameWasReaddressed:
_logger.LogInformation($"{nameof(StreetNameWasReaddressed)} found! Offset: {result.Offset}");

var teHeradresserenLocatiesMessages =
await _teHeradresserenLocatiesMapper.ForAddress(streetNameWasReaddressed.ReaddressedHouseNumbers,
idempotenceKey);

foreach (var teHeradresserenLocatiesMessage in teHeradresserenLocatiesMessages)
{
await _sqsClientWrapper.QueueReaddressMessage(teHeradresserenLocatiesMessage);
}
await _teHernummerenStraatEventProcessor.Process(
TeHernummerenStraatFactory.From(streetNameWasReaddressed), idempotenceKey);

break;

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
namespace AssociationRegistry.Admin.Api.GrarSync;
namespace AssociationRegistry.Admin.Api.GrarConsumer.Kafka;

using Marten.Schema;

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
namespace AssociationRegistry.Admin.Api.GrarConsumer.Kafka.StraatHernummering;

using Be.Vlaanderen.Basisregisters.GrAr.Contracts.AddressRegistry;
using Grar.GrarUpdates.Hernummering;

public static class TeHernummerenStraatFactory
{
public static TeHernummerenStraat From(StreetNameWasReaddressed streetNameWasReaddressed)
{
var result = new List<TeHernummerenAdres>();

foreach (var readdressed in streetNameWasReaddressed.ReaddressedHouseNumbers)
{
result.AddRange(readdressed.ReaddressedBoxNumbers.Select(From));
result.Add(From(readdressed.ReaddressedHouseNumber));
}

return new(result);
}

private static TeHernummerenAdres From(ReaddressedAddressData readdressedAddressData)
=> new(
readdressedAddressData.SourceAddressPersistentLocalId,
readdressedAddressData.DestinationAddressPersistentLocalId);
}

This file was deleted.

This file was deleted.

21 changes: 0 additions & 21 deletions src/AssociationRegistry.Admin.Api/GrarSync/LocatieFinder.cs

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,13 +1,17 @@
namespace AssociationRegistry.Admin.Api.Infrastructure.AWS;

using Acties.GrarConsumer.HeradresseerLocaties;
using Amazon.SQS;
using Grar.HeradresseerLocaties;
using Framework;
using Grar.GrarUpdates.Fusies.TeHeradresserenLocaties;
using Hosts.Configuration;
using Hosts.Configuration.ConfigurationBindings;
using Kbo;
using System.Text.Json;

public class SqsClientWrapper


public class SqsClientWrapper : ISqsClientWrapper
{
private readonly IAmazonSQS _sqsClient;
private readonly string _kboSyncQueueUrl;
Expand All @@ -20,7 +24,12 @@ public SqsClientWrapper(IAmazonSQS sqsClient, AppSettings appSettings, GrarOptio
_readdressQueueUrl = grarOptions.Sqs.GrarSyncQueueUrl;
}

public async Task QueueReaddressMessage(TeHeradresserenLocatiesMessage message)
public async Task QueueReaddressMessage(HeradresseerLocatiesMessage message)
{
await QueueMessage(message);
}

public async Task QueueMessage<TMessage>(TMessage message)
{
await _sqsClient.SendMessageAsync(
_readdressQueueUrl,
Expand All @@ -29,9 +38,6 @@ await _sqsClient.SendMessageAsync(

public async Task QueueKboNummerToSynchronise(string kboNummer)
{
await _sqsClient.SendMessageAsync(
_kboSyncQueueUrl,
JsonSerializer.Serialize(
new TeSynchroniserenKboNummerMessage(kboNummer)));
await QueueMessage(new TeSynchroniserenKboNummerMessage(kboNummer));
}
}
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
namespace AssociationRegistry.Admin.Api.Infrastructure.Extensions;

using Formats;
using GrarSync;
using GrarConsumer;
using GrarConsumer.Kafka;
using Hosts.Configuration.ConfigurationBindings;
using JasperFx.CodeGeneration;
using Json;
Expand Down
Loading
Loading