Display data from massive amount of devices

Universal Dashboard Universal Dashboard Help
1

Hello!

I’d like to use UD for displaying some data collected from ethernet switches and devices, connected to these switches. I have about 50 switches and more than 1500 devices connected. Due to this massive amount of data, I’d like to collect using multiple threads (I already made a script to query and collect data).

What is the best way to organize endpoints for such amount of data? I’m thinking about creating one EP for each switch and cache data, but I need to know if these endpoints will run in different threads simultaneously.

And what do you advise to collect data from devices? Query is just simple tcp request returning json data. Now I use 20 threads and I am able to perform data collection in 30-40 seconds on VM with 1 Xeon core.

1 Like
2

Well this may seem a fudge but if I am collecting lots of data and those queries take a long time. Then I set up a powershell schedule task to run at certain intervals in the day output to a file hosted in same directory as dashboard then display the data from those files.

3

When I run into issues like that, I create a SQL table for the data.
Much better to work with.

1 Like
4

Thank for suggestion, but I think it will be better to put all in one script, IF UD can retrieve data with multiple threads

5

Yes, SQL is much better, but my problem is how to retrieve data, not how to store it

6

I would just make one endpoint, and just use some powershell parallel in that single on.
you could do that in 50 threads.
like this: https://gallery.technet.microsoft.com/scriptcenter/Run-Parallel-Parallel-377fd430

but again, your talking about massive data? what is massive?
do you need to return this massive data?

7

If retrieving the data takes a little longer you could use a scheduled endpoint. If you display the data in a grid you could also use the server side processing approach.