nordigen import failing in the middle

[piotrcichosz]

Support guidelines

• I've read the support guidelines

I've found a bug and checked that ...

• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem
• ... it's definitely a Firefly III issue, not me

Description

last entries in logs:
[2023-04-12 11:03:26] local.DEBUG: [285/285] Done parsing transaction. [2023-04-12 11:03:26] local.DEBUG: Done parsing transactions. [2023-04-12 11:03:26] local.DEBUG: Generated 285 Firefly III transactions. [2023-04-12 11:03:26] local.DEBUG: Filtered down to 285 Firefly III transactions. [2023-04-12 11:03:26] local.DEBUG: Conversion routine "nordigen" was started successfully. [2023-04-12 11:03:26] local.DEBUG: Conversion routine "nordigen" yielded 285 transaction(s). [2023-04-12 11:03:26] local.DEBUG: Transactions are stored on disk "jobs" in file "conv-RxJ44fYc4WZP.json" [2023-04-12 11:03:26] local.DEBUG: Now in setConversionStatus(conv_done) [2023-04-12 11:03:26] local.DEBUG: Found "conv-RxJ44fYc4WZP" in the session [2023-04-12 11:03:26] local.DEBUG: Now in startOrFindConversion(conv-RxJ44fYc4WZP) [2023-04-12 11:03:26] local.DEBUG: Conversion status is "conv_running" [2023-04-12 11:03:26] local.DEBUG: Now in storeConversionStatus(conv-RxJ44fYc4WZP): conv_done [2023-04-12 11:03:26] local.DEBUG: Set conversion as complete.

webpage:
Data conversion
The conversion could not be started, or failed due to an error. Please check the log files. Sorry about this :(

I can try again but then I might run into exceeding limits, is there any option to continue this exact import from conv-RxJ44fYc4WZP.json file?

Debug information

file_exists(): open_basedir restriction in effect. File(/var/www/counter-main.txt) is not within the allowed path(s): (/usr/home/maur/domains/fi.redinn.pl/public_html:/tmp:/usr/share:/usr/local/share:/dev)
after commenting out code for checking txt files in /var/www..

Debug information generated at 2023-04-12 00:00:00 Europe/Warsaw for Firefly III version 6.0.7.

Scope	Version
Firefly III	6.0.7
Firefly III API	2.0.1
Build	(unknown), (unknown)
DB version	19 (exp. 19)
PHP	8.2.4
Host	FreeBSD

System info	Value
System TZ	Europe/Warsaw
Browser TZ	Europe/Warsaw
App environment	local
App debug mode	false
App cache driver	file
App logging	notice, stack
Display errors	Off
Error reporting	ALL errors
Max upload	2147483648 (2147483648 bytes)
Interface	fpm-fcgi
Default language	en_US
Default locale	equal
BCscale	12
DB drivers	mysql, pgsql, sqlite, sqlsrv
Current driver	mysql
Trusted proxies (.env)	``

User info	Value
Login provider	eloquent
User guard	web
Headers	N/A, N/A
Stateful domains
Session start	2023-04-01 00:00:00
Session end	2023-04-30 23:59:59
Session first	2023-01-01 00:00:00
User ID	1
User language	en_US
User locale	en_US
Attempt at "en_US.utf8"	false
Attempt at "en_US.UTF-8"	'en_US.UTF-8'
User agent	Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/110.0.0.0 Safari/537.36

Expected behaviour

import finished and transactions appear in firefly

Steps to reproduce

1. select country, bank, load bank config
2. click next
3. click next

Additional info

No response

[JC5]

What are the logs from the Firefly III side?

[piotrcichosz]

upon refreshing page:
[2023-04-12 11:13:50] local.DEBUG: Now in App\Http\Controllers\Import\ConversionController::index [2023-04-12 11:13:50] local.DEBUG: Will now verify configuration content. [2023-04-12 11:13:50] local.DEBUG: Pressing "back" will send you to mapping. [2023-04-12 11:13:50] local.DEBUG: Will redirect to submission after conversion. [2023-04-12 11:13:50] local.DEBUG: Create Nordigen routine manager. [2023-04-12 11:13:50] local.DEBUG: Conversion routine manager identifier is "conv-RxJ44fYc4WZP" [2023-04-12 11:13:50] local.DEBUG: Stored "conv-RxJ44fYc4WZP" under "conversion_job_id" [2023-04-12 11:13:50] local.DEBUG: Now at App\Http\Controllers\Import\ConversionController::status [2023-04-12 11:13:50] local.DEBUG: Now at App\Http\Controllers\Import\ConversionController::status(conv-RxJ44fYc4WZP) [2023-04-12 11:13:50] local.DEBUG: Now in startOrFindConversion(conv-RxJ44fYc4WZP) [2023-04-12 11:13:50] local.DEBUG: Conversion status is "conv_done" [2023-04-12 11:13:54] local.DEBUG: Now in App\Http\Controllers\Import\SubmitController::index [2023-04-12 11:13:54] local.DEBUG: Submit (import) routine manager identifier is "conv-RxJ44fYc4WZP" [2023-04-12 11:13:54] local.DEBUG: Stored "conv-RxJ44fYc4WZP" under "import_job_id" [2023-04-12 11:13:55] local.DEBUG: Now at App\Http\Controllers\Import\SubmitController::status(conv-RxJ44fYc4WZP) [2023-04-12 11:13:55] local.DEBUG: Now in startOrFindJob(conv-RxJ44fYc4WZP) [2023-04-12 11:13:55] local.DEBUG: Try to see if file exists for job conv-RxJ44fYc4WZP. [2023-04-12 11:13:55] local.DEBUG: File does not exist or error, create a new one. [2023-04-12 11:13:55] local.DEBUG: Return status. {"status":"waiting_to_start","errors":[],"warnings":[],"messages":[]}

(file does not exists? it exists..
ls -l ./storage/jobs/conv-RxJ44fYc4WZP.json -rw-r--r-- 1 user 1000 221081 12 kwi 11:03 ./storage/jobs/conv-RxJ44fYc4WZP.json

however now, after hitting refresh in browser, clicking next/import proceeds with importing

logs from firefly side:
[2023-04-12T11:04:40.682146+02:00] local.INFO: AUDIT: User visits homepage. (185.75..... ([email protected]) -> GET:http://...pl) [] [] [2023-04-12T11:16:31.931278+02:00] local.INFO: AUDIT: Store new transaction over API.
so there is no log around 11:03 - 11:06 when error occured in importer

so only 2 questions:

1. why webpage displayed error "The conversion could not be started, or failed due to an error. Please check the log files" while logs didn't had any error?
2. why after refreshing logs says file for job don't exists - when it clearly exist and clicking next/import started to processing that file and transactions appeared in firefly iii panel :)

[JC5]

1. If PHP times out because your server is configured at something like 30 seconds, this may happen.
2. I don't know.

[piotrcichosz]

ok, most important it works when refreshing :) I will check timeout config

[piotrcichosz]

1. I can't repeat this