add Python.NET recipe #159

Livius90 · 2022-11-12T02:07:46Z

Python.NET (pythonnet) is a package that gives Python programmers nearly seamless integration with .NET Framework, .NET Core and Mono runtime on Windows, Linux and macOS. Python.NET provides a powerful application scripting tool for .NET developers. Using this package you can script .NET applications or build entire applications in Python, using .NET services and components written in any language that targets the CLR (C#, VB.NET, F#, C++/CLI).
http://pythonnet.github.io/

Python.NET uses flit-core package for installing. Flit-core is supported since Yocto kirkstone release.

Livius90 · 2022-11-12T02:24:56Z

@ajlennon
Can you help and share some info about your proxy in your test build? In pythonnet and clr-loader recipe there is a "dotnet restore" which can not work well in your test build. I got this same issue in my Yocto project, i could solve it with the following proxy variable defines.

ajlennon · 2022-11-12T12:16:01Z

Hi @Livius90 I'm not really following I'm afraid as there shouldn't be a proxy in place in the CI build as far as I know?

ajlennon · 2022-11-12T12:24:48Z

Ah! I recall something about this now. Yocto now disallows internet access by default during compile steps unless it is specifically configured to support this. There are some notes in an issue here but I can't find them at the moment

Livius90 · 2022-11-12T21:12:01Z

As i understand nuget package manager is integrated in dotnet, pythonnet and clr-loader .NET projects like to trigger a "dotnet restore" command to restore all of nuget packs from internet. Is it possible somehow let it to get an internet connection? In my local company machine i got the same issue but it was caused by my incorrect proxy settings, now it is different a little in your CI build test.

Livius90 · 2022-11-17T22:25:39Z

@ajlennon
I tested your proxy setting with the following commit: 76a6a45
As i see, you use a docker for build-and-test action and in bitbake log http_proxy and https_proxy env variables are totally empty, it means dotnet restore can not use a correct proxy settings for downloading nugets.

Do you know what should be these proxy variables for your docker?

That same issue was in my company machine too, i could solve it nicely by a pre-defined DOTNET_HTTP_PROXY and DOTNET_HTTPS_PROXY variable in local.conf.

How to configure proxy for NuGet while using Docker?

ajlennon · 2022-11-18T06:46:12Z

Hi - well we can change the envvars if you think it'll help? I still don't really understand why it would be needed but we could give it a go. I would think we could set these in the GitHub YAML. I'm out diving in Egypt until next week but maybe could could try a PR with that which the CI could build in the meantime ?

Livius90 · 2022-11-18T18:27:05Z

I do not know what is your proxy and docker details. I can try to set http_proxy and https_proxy variables to an url value what you can give me first. But probably need any further proxy settings in your docker config too before.

ajlennon · 2022-11-19T07:25:01Z

Well this is what I don't understand. I have no proxy setup and it all works fine... the container access the internet constantly to download source...

Livius90 · 2022-11-19T12:59:19Z

In my company's Linux machine i got http_proxy=http://127.0.0.1:3128 in default Linux environment because we have an installed NTLM. Inside bitbake, dotnet restore was failed always for me also with the same errors but when i did dotnet restore manually in my terminal, it works, magically. I know that this local proxy IP just forwarding the proxy request to an http://my.copmany.com:3128 url, finally i tried to override http_proxy and https_proxy to this direct domain url and it was started to work well for dotnet restore command in bitbake task, too.

There are some many really similar problem in GitHub issues, there are some hints to change any other NUGET env variables to solve it. I will try these things soon.
NuGet/Home#6978

ajlennon · 2022-11-26T12:06:55Z

There is no proxy... the container accesses the internet directly...

Livius90 · 2022-11-27T03:00:55Z

There is no proxy... the container accesses the internet directly...

Just for try i put a webcontent downloading in bitbake task via curl and it seems, internet access is not healthy in your docker, curl can not reach api.nuget.org where the nuget package sources are available.

curl: (6) Could not resolve host: api.nuget.org

https://github.com/DynamicDevices/meta-mono/actions/runs/3555147656/jobs/5971666960#step:7:11572

Is DNS configured well in your docker?
microsoft/vscode-dev-containers#1471

Multiple ethernet LAN/WiFi card in your PC can cause it also.
docker/for-win#2760

ajlennon · 2022-11-27T11:04:44Z

Is DNS configured well in your docker?

Well I am having a look but I can't see how it's possible that DNS is generally broken as the docker container is constantly downloading lots of source files from the Internet...

ajlennon · 2022-11-27T11:13:39Z

OK so I have entered the docker container and I can see I can retrieve the json file OK...

Livius90 · 2022-11-27T14:10:15Z

Hm, it means somewhy in the bitbake compiler task environment the network accessability is different somehow then in your normal docker terminal console has. Possibble it is a docker vs. Yocto bitbake vs. dotnet triple network issue, there are some issue tickets with the similar problem, there were some hints to use hard DNS address and other ideia was to set a metric/priority settings for multiple ethernet LAN connections, if it has, but unfortunately these issues was closed without final solution.

That problem is realy big issue because a Yocto recipe can use dotnet-native for any simple cross-compilation for .NET 6.0 apps but if their apps have any nuget depedency it can be easily failed in a docker container. For me in my phisical copmany's Ubuntu Linux machine i got only a proxy missconfiguration problem, i could solve it quickly via the http_proxy environment variables. Probably on any non-proxy and non-docker building Linux machines can use well dotnet-native in a recipes but this situation is not so nice, if it randomly can not work or can work in a docker containers.

ajlennon · 2022-11-28T12:54:33Z

SyntaxError: unexpected character after line continuation character

Livius90 · 2022-11-28T17:20:47Z

I could not leave DOTNET_HTTP_PROXY definition in my last test. My company's proxy work well only if i pre-define it in my local.conf. Usage of do_compile[network] = "1" does not matter for it.

Livius90 · 2022-11-29T22:24:17Z

@ajlennon
How can we add a custom unit-test recipe for pythonnet on qemu machine target? We can execute the .NET core tests from pythonnet source easily like pythonnet action/workflow do it here.

Need to just execute the following command in tests folder of pythonnet repo.

python3 -m pytest --runtime coreclr

ajlennon · 2022-11-30T13:09:41Z

Merged in. That's great. We got there in the end and it also validates the whole CI testing setup for me.

You can probably enable unit testing on these recipes as Yocto supports this.

The other way is to add some test classes as I've done for some specific simple tests which will be run during the CI build in QEMU

e.g.

https://github.com/DynamicDevices/meta-mono/tree/master/lib/oeqa/runtime/cases

* dotnet: update to v6.0.11 * python: Add Python.NET v3.0.1 * python3-clr-loader: fix typo * pythonnet: fix proxy config * pythonnet: separated folder for nuget downloads and cache * python3-clr-loader: test proxy * dotnet restore try to fix DynamicDevices#1 * dotnet restore next try * dotnet restore try some http env settings * dotnet restore fix libcurl-native syntax * dotnet restore curl fix * dotnet restore test http download * dotnet restore http next try * dotnet restore test DNS * dotnet restore, dns fix not works, revert some parts * dotnet restore fix, version appending fix in __init__.py * fix endline, restore original yml * fix again endline * fix again endline and trigger a new fetch * append endline in do_patch, restore latest package versions * do_configure:prepend() must be used for version append * append __version__ only if it is does not exist * fix typo * if proxy used, it must be defined in task environment

Livius90 added 4 commits November 9, 2022 11:17

dotnet: update to v6.0.11

c77a1af

Merge branch 'DynamicDevices:master' into master

cccad24

python: Add Python.NET v3.0.1

313b3dd

python3-clr-loader: fix typo

5872fdb

pythonnet: fix proxy config

57389e3

Livius90 added 2 commits November 14, 2022 00:36

pythonnet: separated folder for nuget downloads and cache

fa62409

python3-clr-loader: test proxy

76a6a45

Livius90 added 5 commits November 20, 2022 22:38

dotnet restore try to fix #1

1195469

dotnet restore next try

fed7aad

dotnet restore try some http env settings

6624e00

dotnet restore fix libcurl-native syntax

099ebb0

dotnet restore curl fix

8f49947

Livius90 added 2 commits November 26, 2022 20:41

dotnet restore test http download

0c251a9

dotnet restore http next try

d650837

Livius90 added 2 commits November 27, 2022 04:17

dotnet restore test DNS

b92326e

dotnet restore, dns fix not works, revert some parts

0d89725

Livius90 added 6 commits November 27, 2022 17:12

dotnet restore fix, version appending fix in __init__.py

85ed263

fix endline, restore original yml

a571e81

fix again endline

7424f94

fix again endline and trigger a new fetch

6ffbb6c

append endline in do_patch, restore latest package versions

4383f75

do_configure:prepend() must be used for version append

38c93c7

Livius90 mentioned this pull request Nov 27, 2022

Raising my own PR on #159 to look at the python3 build problem independently #160

Closed

Livius90 added 2 commits November 28, 2022 00:29

append __version__ only if it is does not exist

c3bb690

fix typo

7f96e54

if proxy used, it must be defined in task environment

a99642d

ajlennon merged commit 35d7818 into DynamicDevices:master Nov 30, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add Python.NET recipe #159

add Python.NET recipe #159

Livius90 commented Nov 12, 2022

Livius90 commented Nov 12, 2022

ajlennon commented Nov 12, 2022

ajlennon commented Nov 12, 2022

Livius90 commented Nov 12, 2022 •

edited

Loading

Livius90 commented Nov 17, 2022 •

edited

Loading

ajlennon commented Nov 18, 2022

Livius90 commented Nov 18, 2022 •

edited

Loading

ajlennon commented Nov 19, 2022

Livius90 commented Nov 19, 2022 •

edited

Loading

ajlennon commented Nov 26, 2022

Livius90 commented Nov 27, 2022 •

edited

Loading

ajlennon commented Nov 27, 2022

ajlennon commented Nov 27, 2022

Livius90 commented Nov 27, 2022 •

edited

Loading

ajlennon commented Nov 28, 2022

Livius90 commented Nov 28, 2022

Livius90 commented Nov 29, 2022 •

edited

Loading

ajlennon commented Nov 30, 2022

add Python.NET recipe #159

add Python.NET recipe #159

Conversation

Livius90 commented Nov 12, 2022

Livius90 commented Nov 12, 2022

ajlennon commented Nov 12, 2022

ajlennon commented Nov 12, 2022

Livius90 commented Nov 12, 2022 • edited Loading

Livius90 commented Nov 17, 2022 • edited Loading

ajlennon commented Nov 18, 2022

Livius90 commented Nov 18, 2022 • edited Loading

ajlennon commented Nov 19, 2022

Livius90 commented Nov 19, 2022 • edited Loading

ajlennon commented Nov 26, 2022

Livius90 commented Nov 27, 2022 • edited Loading

ajlennon commented Nov 27, 2022

ajlennon commented Nov 27, 2022

Livius90 commented Nov 27, 2022 • edited Loading

ajlennon commented Nov 28, 2022

Livius90 commented Nov 28, 2022

Livius90 commented Nov 29, 2022 • edited Loading

ajlennon commented Nov 30, 2022

Livius90 commented Nov 12, 2022 •

edited

Loading

Livius90 commented Nov 17, 2022 •

edited

Loading

Livius90 commented Nov 18, 2022 •

edited

Loading

Livius90 commented Nov 19, 2022 •

edited

Loading

Livius90 commented Nov 27, 2022 •

edited

Loading

Livius90 commented Nov 27, 2022 •

edited

Loading

Livius90 commented Nov 29, 2022 •

edited

Loading