ElasticSearch Mapping Visitor Pattern Not Applied on Dynamic Object

The joy of learning new technology is learning new constraints. There is never enough documentation.

Problem

I need to insert dynamic objects into ElasticSearch. For the sake of consistency, I need some properties mapped to specify types and the rest will be mapped text. One amazing thing is the ElasticSearch automagical conversion called AutoMap. Seems like AutoMap with visitor pattern and properties override will meet my requirements, so I have something like this in my code:

public class MapToTextPropertyVisitor : NoopPropertyVisitor
{
    public override IProperty Visit(PropertyInfo propertyInfo,
            ElasticsearchPropertyAttributeBase attribute) => new TextProperty();
}

var createIndexResponse = _client.Indices.Create("<index_name>", c => c
    .Map<dynamic>(m => m
        .AutoMap(new MapToTextPropertyVisitor())
        .Properties(p => p.Date(d => d.Name("<overrides_date_field>"));

AutoMap with manual overrides using fluent mapping:
https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/fluent-mapping.html#_auto_mapping_overrides_down_the_object_graph

Visitor pattern: 
https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/visitor-pattern-mapping.html

But for somewhat reason, some of my dynamic object's properties that are not overridden are still mapped to date type when it is supposed to be mapped to text.

Solution

I found out later that the visitor pattern doesn't apply to dynamic mapping in which my dynamic objects are subjected to. It only applies to POCO with clear types. 

Another thing is in my case, the properties in my dynamic object are all of type string. String in ElasticSearch for dynamic mapping has two different kind of detections applied to it:

• Date detection
• Number detection

https://www.elastic.co/guide/en/elasticsearch/reference/current/dynamic-field-mapping.html

So, my problem were solved by disabling the two detections. That makes the string properties stay as string.

Amazon ECR Accessing Private Repository through AWS CLI

 Remote repository is always an easy way to share code. However, I can't seem to find an easy way to access my private repository in ECR.

Problem

In order to push or pull from ECR, we have to first login via AWS CLI and pass the credentials to docker. The script is pretty straightforward and will work in most cases except mine:

aws ecr get-login-password --region <region> | sudo docker login --username AWS --password-stdin <registry_url>

Solution

The problem is I save my credentials for the registry in a different AWS CLI profile, thus I need to change my script to the following so I can push to and pull from the private repository:

aws ecr get-login-password --region <region> --profile <custom_profile> | sudo docker login --username AWS --password-stdin <registry_url>

FTP Access Denied When Attempting to Transfer Files

Permission is, as usual, a double-edged sword. It happened when I tried to transfer files to a remote Linux server.

Problem

I created a directory on the remote server and attempting to transfer files via FTP client but got an access denied error. It is weird because I do login using the same account and thus as the owner of the directory.

Solution

The one that solves the problem in my case is by changing ownership of the folder recursively (-R option) and use name of the user as group name such as:

sudo chown -R <user>:<user> <directory>

React Navigation Handle Header Button Click on Child Screen or Component

I have a react native application which I need to handle header button / action button click on the detail screen. And the journey to solve it was not a short one.

Problem

On the parent screen, I have defined a stack navigation and set the header button. Then I need to handle the header button click on the child screen. I read about passing the function as parameter but it's not easy. There are also options to useEffect or ref, but none of them are working for me.

Solution

In the end, the one that works for me and pretty clean too is by using React.useLayoutEffect and navigation.setOptions:

const ChildScreen = ({navigation, route}) => {
  React.useLayoutEffect(() => {
    navigation.setOptions({
      headerRight: () => (
        <Button onPress={() => ...} title="Right" />
      ),
    });
  }, [navigation]);
  
  return (...)
};

Reference:

https://reactnavigation.org/docs/header-buttons/#header-interaction-with-its-screen-component

MSBuild Copy Task AfterBuild VS2019

This time the issue is with MSBuild task that I set up in one of my project (in the project file). The task is to copy the dll to a different location after build is done. I copied the configuration over to a different project and guess what, it didn't work.

Problem

It worked flawlessly for a very long time with the following configuration:

<Target Name="AfterBuild">
  <Copy SourceFiles="..." DestinationFolder="..." />
</Target>

Solution

Two main differences between the two projects are the working on is a .NET Framework project and built in VS2017. The new one is a .NET Standard and built in VS2019. Apparently, there is a change for VS2019 that comes with updated MSBuild. It is no longer depends on the target name (it is a bad idea anyway) to determine when to execute the task. I update it to the following and then it works great.

<Target Name="AnyNameIsFine" AfterTargets="Build">
  <Copy SourceFiles="..." DestinationFolder="..." />
</Target>

For reference:

https://docs.microsoft.com/en-us/visualstudio/msbuild/copy-task?view=vs-2019

NuGet Package Reference NU6105 Publish Error

Some of my .NET Core applications are already using PackageReference which is a very nice idea. However, through a combination of packages, Visual Studio did not allow me to publish my project although it built fine.

Problem

During publish, it threw error on NU6105 warning. Along with that most of it comes with the following message:

Detected package downgrade

Solution

Some developers solve it by finding which package caused the issue and manually added them through NuGet, but I find them troublesome until I found the following article:

https://docs.microsoft.com/en-us/nuget/reference/errors-and-warnings/nu1605

In my case, all I need to do to solve it is to install the following NuGet package: 

Microsoft.NETCore.Targets

SNS HTTPS Fanout to API Gateway Error

 In one of my projects, I have an Amazon SNS subscription set up to fan out to HTTPS endpoint that is backed by Amazon API Gateway. The API Gateway has mapping template applied. 

Problem

It went smoothly during test with Postman and the API Gateway test, but when SNS sends the Notification message, it threw an error. On CloudWatch log, the error message is:

Execution failed: null

Solution

Apparently SNS sends request with Content-Type of text/plain although the request body contains json while I only had mapping set for application/json. So adding mapping template for text/plain solves my problem.

Unwanted Dollar Sign in Bash Script

The script that I used ran fine for various flavors of Linux for a long time. But somehow, it produced unwanted dollar sign on Ubuntu 18.04, so my hours of troubleshooting starts.

Problem

The simplest one I can say is I use echo with tab and variables and piped that to awk through AWS Systems Manager. It is similar to the following:

v='variable'
TAB=$'\t'
echo "${v}${TAB}" | awk '{print $0}'

In many flavors of Linux other than Ubuntu 18.04, even in Ubuntu 16.04, it produced the expected result:

variable

However, in Ubuntu 18.04, it ends the result with an extra dollar sign:

variable$

At first, I thought it marks end of line or the typical \0 (NUL character) that marks end of string, so I tried various ways to remove it such as using tr, gsub, etc. But none of them works.

Solution

Eventually, I found out that the dollar sign comes from the TAB variable. To solve it, I have to replace the above with:

echo -e "${v}\t"

Searching Files by DateModified in File Explorer in Windows

I was in the middle of some project files restructuring and part of the process is making a back up of my files. When all is done, I went back to my back up folder to find files that I modified a day before.

Problem

I know there has to be a way to do that, but it is not immediately obvious. But the following article was helpful to me.

https://www.howtogeek.com/243511/how-to-search-for-files-from-a-certain-date-range-in-windows-8-and-10

I opted for the UI solution, that is to use the Search tab. However, I can't find the search tab in File explorer, so I decided to go the harder route, to type into the search box.

Solution

I managed to file my files by typing the following in the search box:

datemodified:<start_date>..<end_date>

For example:

datemodified:9/14/2020..9/15/2020

modified: instead of datemodified: works too. Also, the search tab finally shows after I got my search result.

Signing Certificate is None in Xcode

Xcode has this capability to manage certificates, app ID and provisioning profile which makes is very convenient. However, this time, I would like to manage my own.

Problem

I manage to get Xcode to recognize the provisioning profile that I have created in developer.apple.com. However, under .xcodeproj (project file) > Signing & Capabilities > Signing Certificate, the value is none. It also comes with an error that it the provisioning profile is not associated with its own developer certificate. I have made sure that the certificate associated with the provisioning profile is stored in my Keychain, but it seems like Xcode is still trying to use a different certificate for signing.

Solution

My guess was right, I went to .xcodeproj (project file) > Build Settings > Code Signing Identity and found out that it is set to iOS Developer under "Automatic" section. Switching it to the right (associated with provisioning profile) certificate on the Keychain solves the issue.

Xcode CodeSign Incorrectly States Password is Incorrect

I was trying to create an archive in Xcode to prepare the app for testing. It requires a code signing certificate and Xcode tried to reach out to Keychain to obtain it, so it prompted me for a password to my Keychain.

Problem

My first thought is since the Keychain lives in my Mac, it has to be my Mac's password, so I entered it and it failed. I re-checked the characters, re-entered and it still failed. That is odd considering I did that before without issue.

Solution

I found out that my Keychain Access app was still active. I have to quit the app and then it proceeds properly. So, the solution is quit the Keychain Access app before entering password for signing.

Amazon CloudWatch GetMetricData API SampleCount Returns 0

One of our code is to keep track on how many data points are there for each CloudWatch metric. So, I usually use the GetMetricData API with statistic set to SampleCount. But since we only care about the total, I don't use short period (high resolution). Partly, it is to reduce the amount of data returned. However, in one case, it returns 0 for a particular month.

Problem

According to the following documentation, period has to be a multiply of 60:

https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_MetricDataQuery.html

No problem, since my start time and end time is exactly one month apart, I will just set the period to the number of seconds between start time and end time to get highest number of period. It works for some months, it returns 0 in this particular case.

Solution

Strange until I found out in a different documentation that there's a max period of 86,400 seconds (one day).

https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html#Statistic

I change my period to 86400 and it no longer returns 0.