Hello Scott,

Really sorry to see that you are experiencing these
issues. This is definitely not expected behavior based on
your description of what you were editing.

In regards to your original instance;

Is there a chance that you have another instance of Tomcat
running in the background and using the same port?
I would confirm you can telnet to the server via ?Telnet DB port?
This may shed some light on what is happening with this port. Depending
on the outcome, it might worth changing the port number for that instance.

In regards to your issues with failed imports into a new instance;

Is it possible to send the YF log files through that cover that issue
so we can see what is happening?

Lastly, what version and build of YF are you using? E.g. 7.1, Dec?

Thank you,

Kyle

sorry not an answer a reply/comment

Hi Scott,

Thank you for the prompt response and for sending the
additional log files.

We will take a look them and see what we can find out.

Please let us know if your admin is able to find out
anything more regarding the ports.

Regards,

Kyle

Hey Scott,

I got some feedback from other team members. It's probably
best that you zip up all the YF log files.

Just grab all files under Yellowfinappserverlogs and send them across to
support@yellowfin.bi - you can reference this title of this post as well.

Thank you very much,

Kyle

ok cheers

we cleared port error - but we are still concerned we cant export and import artefacts....

8080 was used in server_tomcat.xml too
[9:48:01 AM] Scott Hyde: but what would have changed this?
[9:48:03 AM] israr ahmed khan: i have changed server_tomcat port to 8085
[9:48:10 AM] israr ahmed khan: and server.xml was 80


change made by server admin

any update here please?

Hello Scott,

I'll report back to you when I hear more from Senior Support
regarding your latest logs.

Generally, for any forum requests we aim to respond anywhere
from 48-72 hours, this would not include weekend days.

Can you confirm that you now able to successfully run you instance
from the stop / start service??

Thank you,

Kyle

thanks for follow up

Yes we can stop start/service now after removing the second xml file

I have no idea where it came from?


thanks!

Still have issues with Import and Export

Hi Scott,

thanks for the logs, I found the following error during an attempt to export:

[code]
2015-02-05 11:58:29 ERROR: [269339] Connection[0] checkout timeout exceeded
2015-02-05 11:58:29 ERROR: [269339] Connection[0] was checked out at 2015-02-05 11:55:29
2015-02-05 11:58:29 ERROR: [269339] at com.hof.pool.DBConnectionManager.getConnection(DBConnectionManager.java:281)
2015-02-05 11:58:29 ERROR: [269339] at com.hof.util.DBAction.(DBAction.java:333)
2015-02-05 11:58:29 ERROR: [269339] at com.hof.mi.util.ViewCache.A(ViewCache.java:542)
2015-02-05 11:58:29 ERROR: [269339] at com.hof.mi.util.ViewCache.getChildElements(ViewCache.java:510)
2015-02-05 11:58:29 ERROR: [269339] at com.hof.mi.process.V4ExportProcess.A(V4ExportProcess.java:1826)
2015-02-05 11:58:34 WARN: [269339] Timeout occurred when closing connection: java.util.concurrent.TimeoutException
java.util.concurrent.TimeoutException
at java.util.concurrent.FutureTask.get(Unknown Source)
at com.hof.pool.JDBCConnection.C(JDBCConnection.java:298)
at com.hof.pool.JDBCConnection.closeConnection(JDBCConnection.java:211)
at com.hof.pool.DBConnectionPool.D(DBConnectionPool.java:1538)
at com.hof.pool.DBConnectionPool.A(DBConnectionPool.java:208)
at com.hof.pool.DBConnectionPool$_A.run(DBConnectionPool.java:1849)
at java.util.TimerThread.mainLoop(Unknown Source)
at java.util.TimerThread.run(Unknown Source)
[/code]

which means the Yellowfin connection to the config DB timed out. Please see this other forum post for information on how to increase the checkout time for your data source.

I hope that resolves your export/import issues, if not, then please send us a screenshot of the error, and give us more of a description of when it occurs, and if it is only for a specific content item or all items.

regards,
Dave

Ok will update and try this

We are still have config using trial hsql db

I think we should move this to RDS postgress as we in start up prod - can we do this with current system or do we need to export repo and import artefacts into a system installed and set up this way?

thanks

Ok will update and try this

We are still have config using trial hsql db

I think we should move this to RDS postgress as we in start up prod - can we do this with current system or do we need to export repo and import artefacts into a system installed and set up this way?

thanks

actually failed again at 5 mins

Any idea of the expected time for this process?

Hi Scott,

yes, good idea, definitely use a proper database such as PostgreSQL for the Yellowfin config DB instead of the demo HSQL db, I have seen HSQL DBs get corrupted.

You'll have to export your Yellowfin contents (data sources, views, etc.) via the Export feature and then import them into the new Yellowfin DB.

From your Data Source log I can see that initially your checkout time was 3 mins:

[code]
2015-02-05 11:55:29 NOTICE: [269339] -----------------------------------------
2015-02-05 11:55:29 NOTICE: [269339] 2015-02-05 11:55:29
2015-02-05 11:55:29 NOTICE: [269339] Starting DBConnectionPool
2015-02-05 11:55:29 NOTICE: [269339] sourceId: 269339
2015-02-05 11:55:29 NOTICE: [269339] sourceName: YF config
2015-02-05 11:55:29 NOTICE: [269339] connectionMethod: JDBC
2015-02-05 11:55:29 NOTICE: [269339] authAdapter: Standard Authentication(GENERICUSER)
2015-02-05 11:55:29 NOTICE: [269339] driver: org.hsqldb.jdbc.JDBCDriver
2015-02-05 11:55:29 NOTICE: [269339] url: jdbc:hsqldb:file:C:UsersAdministratorYellowfin 7.1/data/configdb;shutdown=true
2015-02-05 11:55:29 NOTICE: [269339] username: SA
2015-02-05 11:55:29 NOTICE: [269339] min connections: 1
2015-02-05 11:55:29 NOTICE: [269339] max connections: 5
2015-02-05 11:55:29 NOTICE: [269339] maxOpenTime: 3 h
2015-02-05 11:55:29 NOTICE: [269339] maxCheckoutTime: 3 min
2015-02-05 11:55:29 NOTICE: [269339] psCacheSize: 0
2015-02-05 11:55:29 NOTICE: [269339] FetchSize: 0
2015-02-05 11:55:29 NOTICE: [269339] Verify connections: false
2015-02-05 11:55:29 NOTICE: [269339] Application Version: 7.1
2015-02-05 11:55:29 NOTICE: [269339] Build Version: 20140827
2015-02-05 11:55:29 NOTICE: [269339] Pooling Enabled: true
2015-02-05 11:55:29 NOTICE: [269339] -----------------------------------------
[/code]

so you must have increased it to 5 mins, and yet it is still timing out.

I'm afraid I can't offer an estimate to how long you should set the checkout time for. That's in the same category of questions as the famous "how long is a piece of string?" Are you trying to export all of your data sources, views, reports, dashboards at the same time? You could try doubling the current checkout time.

Or, you never know, there might be one item that is causing the system to hang, in which case, you could try importing the content items one by one (or at least in smaller groups).

Please let us know how you get on with this.

regards,
Dave

Yep I was trying to export all so as to move to new system,

That would be normal practice wouldn't it - as a backup strategy?

I suppose I am asking if you see it take 10-20 mins to do this?

if not one item must be the issue....

thanks

Hi Scott,

you can do that as a backup strategy although keep in mind that not everything in the Yellowfin repo can be exported (e.g. user groups, security settings etc.), but by far the most popular, effective (and easiest) backup strategy is to do a dump of the Yellowfin config db.

I have definitely seen one export of all contents take over 10 mins on my machine.

I think it would definitely be worth your while to start off trying to export just one item, because this test won't take you long and it will let us know whether the basic export process is working OK.

Please let us know how it goes.


regards,

Dave

Ok so YF config db ALSO holds structures of reports and dashboard? If all is held in the config db yes - this would be a much easier strategy.

I assumed the config db was user stuff/security only and export provided reports/dashboard artefacts for example...

thanks

yes, you could summarize it by saying that the YF config DB contains everything except for the actual report data. There are one or two exceptions to this, but speaking generally this is a good overall view on what the YF config DB does.

regards,
Dave

Ok great

so is it possible to transfer the data from the HSQL db to a new postgress db and connect to this?

This would therefore remove the need for export/ import correct?

thanks

We roll to some major clients in the next few weeks and really want to get this 100% right :)

Hi Scott,

I don't know, I've never tried. We've had a few clients successfully migrate all of their HSQL db into a MySQL db, but haven't heard of any doing the same with PostgreSQL.

I do know that different databases can do certain things differently and therefore it mightn't be as straightforward as one would think.

I guess the only way to find out would be to give it a go. Please let us know how it goes.


regards,
Dave

Ok so the standard standard or usual migration process from this hsldb to anther db is via the export import into a new installation set up with whatever new db chosen?

yes, if you export and import via Yellowfin then the application and drivers take care of the differences in databases, however you can only handle data sources, views, reports, content categories, dashboards, storyboards with this process, any other metadata such as users, user groups etc. is not able to be exported/imported via the Yellowfin export/import feature.

regards,
Dave

Ok so export continues to fail on batch prior to time out

I can get a few small artefacts to export

Can I supply log file of error again?

If so which file please

Hi Scott,

just zip up the whole logs folder and email it to us, don't forget to quote the subject of this forum post.

Could you please give us more information about the tests you have tried, e.g. you tried to export 1 report by itself and the export worked. Then you tried the complete contents collection and it failed again.

Looking ahead, if there is some sort of a bug with exporting one particular item, then what we'll have to do is to get you to upload your YF config DB and the schema from the datasource that is used by the failing view or report or dashboard (i.e. we don't need the data, just the schema). Then a developer can go through the export process in debug mode over here, stepping through the code line by line to see exactly what's tripping the process up.

regards,
Dave

I cant seem to find any support email address?
Can you provide URL to same please?

Hi Scott,

no worries, the Yellowfin support email address is:

support@yellowfin.bi

regards,
Dave

Cheers thanks

I've emailed in today logs only - a fail occurred today as full log folder 22 meg zipped

Any update here guys??

Still cant export full artefacts without killing server...

thanks

Hi Scott,

sorry but your email didn't come through to us, I think it was too large an attachment. Actually you could just upload it to our FTP site (using that account we created for you in Dec), or I guess you could split the logs into 2 zip files.

regards,
Dave

Hi Scott,

I located the email with you logs attached and forwarded them to Dave so
he can take a look at them.

Sorry for the inconvenience this has caused. Hopefully we can figure out
what is going on after reviewing the logs.

Cheers,

Kyle

Hi Scott,

thanks for the logs, I can see that you are getting a "connection doesn't exist" error message a few minutes after you started trying to export your artefacts, then I noticed from your source logs that your data source have been configured for a maximum timeout of 3 minutes, so was wondering if you could try increasing the length of the data source timeout as described in this wiki page and see if that solves the issue.

I also noticed that your view is cached, now that shouldn't make any difference, however if the above suggestion doesn't help then it might also be interesting to try deleting the view cache and see if exporting the views when they are running from live data helps.

Otherwise, if neither of the above suggestions help then we will have to set up your environment over here so a developer can step through the source code while trying to export and see where it's tripping up. To do this you would need to upload your full Yellowfin config DB and just the schema of your data source (i.e. we don't need your actual data, just a script so we can re-create an empty version of your postgresql data source)

Please let us know how it all goes.

regards,
Dave

Hi David

here is screen dump of my config

it has been set at 10 minutes when you first flagged this as a possible solution.

I am going to clear browser cache and try again now.


Can you explain how I send you backup of config settings as opposed to Export artefacts please..

thanks

Sorry ive just reread

so lost my data source after 3 mins?

So my data sources need more time when exporting structures?

Hi Scott,

I didn't mean the browser cache, I meant the view cache:






also, don't worry about the data source checkout time theory, that was wrong, Yellowfin checks the data source connection when it exports but it doesn't verify the query.

Regarding how to get your Yellowfin config db and data source schema to us:

because you're using the embedded HSQL db as the config db then that is easy...just zip up the data folder and upload it to our FTP website (I've just created an account on it for you - you should receive your login details in an auto-email any minute now).

As for how to create a dump of your data source schema, you just use the pg_dump command as normal, except this time use the -s switch as that signifies to not include the data.

regards,
Dave

I have uploaded 4 dump files that are the schema of any data sources included

I have loaded the db file

I have sent to anonymous - but I see them in my account now..

please advise if all correct



thanks

Hi Scott,

thanks for that, it actually took quite a bit of work to restore them, (just out of interest I have attached to this forum post a document on some of the issues I encountered when trying to restore them) but eventually got there.

I set my YF config DB timeout to 30 mins and tried to export everthing, and got a timeout error. Then rather than setting the timeout to a few hours and then waiting to see if the next full export completed, I thought it would be expedient to just separate the artefacts into groups defined by their object types (DATA SOURCES, VIEWS, REPORTS, etc.), so I gave this a go and it worked fine - everything exported without any errors. So I have emailed you an archive of the resulting export files, it shouldn't take too long to import them, start with the Data Sources, then Views, and so forth.

Please let us know how it goes.

regards,
Dave

Ok thanks for your help here

I have successfully rebuilt reports and dash (critical ones for noe) into a new scaled instance and also one that points to a more redundant postgress AWS RDS datastore...

So given the above comments in previous posts a simple back of this db (which is automatic in RDS) is our back up strategy and we could plug any new instance into this db if a recovery was needed.

that's great news, glad you have been able to import your content into a proper DB such as PostgreSQL!
Yes you're correct about the backup strategy, if you ever lost your postgresql YF config db then you'd just have to restore your auto-backup dump, and because it would have the same name and address as the previous one then you won't even have to re-point Yellowfin to it. However, if for whatever reason you wanted to do a fresh new installation of Yellowfin then the easiest way would be to simply install Yellowfin and choose the HSQL database option, then after the installation is complete then run Yellowfin once (this creates the ROOT folder and sub-folders) and then point Yellowfin to your backup config db by modifying the correct paramters in the web.xml file as described in this other forum post.

An important thing to watch out for is that if you do a fresh new installation of Yellowfin and choose PostgreSQL to install into and de-select the option "Create new database" and point the installer to your backup YF config DB thinking that Yellowfin will use the existing YF config DB as it is...well it won't! the installer will overwrite that existing backup db with all new empty tables, columns etc.

So that's why I mentioned above about installing YF into the HSQL DB (i.e. temporarily) and then after the installation re-pointing YF to your backup DB.

regards,
Dave

All makes sense and thanks for the tip - of create HSQL first

exactly what I would have done !

thanks

system running really well on new instance/db