missing API against TH

The following math functions are missing in THC but present in TH:

numel
prod
zeros
ones
reshape
rand
randn
round
atan2
std (not stdall, this is per-dimension)
var (not varall, this is per-dimension)
cumsum
cumprod
maskedFill
maskedSelect
sort
maskedCopy (waiting for merge, Streams for BLAS. maskedCopy. generic reduce kernels. bugfix. #167)
multinomial (implemented, waiting for PR)
cross
logicalall
logicalany
tril
triu
trace
diag
nonzero
range
cat
linspace
logspace
eye
randperm
histc
conv2 (these are torch operators, not to be confused with nn Convolutions which are already implemented)
conv3

When these are implemented, cwrap entries can be added that would make cutorch completely API compatible with torch

@soumith

Hey @soumith , so running cutorch.test() currently reports errors on cumsum and cumprod (missing). That's on purpose?

Yes they are yet to be implemented on cutorch. I'll get to that next week. They can be done with a thrust:: scan

Ok cool.

On Sat, Nov 8, 2014 at 12:27 PM, Soumith Chintala notifications@github.com
wrote:

Yes they are yet to be implemented on cutorch. I'll get to that next week.
They can be done with a thrust:: scan

—
Reply to this email directly or view it on GitHub
#70 (comment).

Does anyone have an implementation of std and var yet? Otherwise I'll have a go next week.
Afaict, we can't use THCudaTensor_reduceDim so we'll need something custom (but with a similar structure as reduceDim).

no, do not have it. this has been long overdue, but we should co-ordinate these, let me email you guys.

Any progress on cumsum and cumprod?

have not started them yet!

Okay, I might have a go soon then, because we'd like to use it. Just waiting for the THCState PR to be merged.

I will merge the THCState PR on Friday. All the patches have been prepared except for fbcunn, working on that as well.

@soumith

@soumith For reductions along a single dimension, we still have the restriction that the tensor must not have more than 4 dimensions. Did you say you're working on a fix for that? If so, what's the progress on it?
If you're not working on it, we could easily change the code to what we do for std and var, where there's no restriction on dimensionality at all.

@wickedfoo

@wickedfoo already has PR for that internally. Its all implemented. I'm on
vacay till 26th, so I will sync those changes at the end of this month.
On Feb 19, 2015 4:34 PM, "Dominik Grewe" notifications@github.com wrote:

@soumith https://github.com/soumith For reductions along a single
dimension, we still have the restriction that the tensor must not have more
than 4 dimensions. Did you say you're working on a fix for that? If so,
what's the progress on it?
If you're not working on it, we could easily change the code to what we do
for std and var, where there's no restriction on dimensionality at all.

Reply to this email directly or view it on GitHub
#70 (comment).

His PR is for apply (and apply2) along an arbitrary dimension. Not reductions. It generalizes the copy kernels and changes all the tensor math to use these apply kernels where appropriate instead of make contiguous + thrust

That's the status on that. He did not work yet on arbitrary reductions. If you want to tackle that, go for it.

Thanks for the update. I'll have a go at the reductions kernel then.
Looking forward to the apply kernels!

That's right. I meant to do it, but haven't had the time yet, sorry.

For gather, which I suddenly realize could be useful for implementing ClassNLLCriterion.forward, without needing a custom kernel, I guess?, I suppose a simple naive first-cut could be:

use the isContiguous (toContiguous? asContiguous?) method to convert the tensor to contiguous format
simply assign one thread to each output location, and I dont think we need any local memory or anything right? So, just throw everything into warp-size blocks and it's basically done?

Does that sound about right? Anything else I should bear in mind if I write a naive gather along these lines? (I'll be targeting cltorch I confess, but it's quite easy to convert cutorch<->cltorch kernels I think?)

(Edit: what do you think is the most similar existing class/kernel to base this off? and/or thoughts on where to put this, ie filename(s)?)

@SamGross

One of my colleagues @SamGross is working on it by interfacing the magma cuda library. It'll happen over the next month or so when he finishes it up and sends a PR.

Magma looks cool. Has opencl version too it seems :-)

Here is a gather implementation. It's not very tested yet..

The kernel: https://github.com/hughperkins/cltorch/blob/master/lib/THCl/THClGather.cl
The driver: https://github.com/hughperkins/cltorch/blob/master/lib/THCl/THClGather.cpp
The lua wrapper: https://github.com/hughperkins/cltorch/blob/master/torch/generic/Tensor.c#L539

Shoe-horned the lua wrapper into TensorMath.lua: hughperkins/cltorch@0e469f4

Did scatter too, since seems like more of the same?

(Edit: and scatterFill, in same files)

Is there any update on adding these functions? I'm in need of the cross product on cudatensors.

I can code it up if someone can point me towards the things that need to be done. All the layers I've written so far are on the lua side and I'm not sure how to make the connection between cuda and lua. Thanks!

@abyravan

@abyravan i could get to cross next week.

That would be great. Thanks a lot! Is there any sort of a tutorial or howto on adding new functionality for tensors? Would be useful to have :)

You can look at existing PRs that are cross-linked in this thread.
Like:
#120
#96
#75

nonzero is being implemented by FB, should be out in a few days.

cutorch

Hi guys,

any hope on implementing the conv2 (or xcorr2 for that matter)?

Just adding a note that eye has been implemented here pytorch/pytorch#2395

Thanks for supporting many of the math functions in THC. I'm now waiting for histc!

Navigation Menu

Repository navigation

missing API against TH #70

25 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

missing API against TH #70

Description

Activity

clementfarabet commented on Nov 8, 2014

soumith commented on Nov 8, 2014

clementfarabet commented on Nov 8, 2014

dominikgrewe commented on Jan 3, 2015

soumith commented on Jan 3, 2015

dominikgrewe commented on Jan 20, 2015

soumith commented on Jan 20, 2015

dominikgrewe commented on Jan 20, 2015

soumith commented on Jan 21, 2015

dominikgrewe commented on Feb 19, 2015

soumith commented on Feb 19, 2015

soumith commented on Feb 19, 2015

soumith commented on Feb 19, 2015

dominikgrewe commented on Feb 19, 2015

25 remaining items

dominikgrewe commented on Jun 15, 2015

hughperkins commented on Jun 23, 2015

hughperkins commented on Jun 23, 2015

hughperkins commented on Jun 23, 2015

hughperkins commented on Jun 24, 2015

hughperkins commented on Jun 24, 2015

abyravan commented on Feb 23, 2016

soumith commented on Feb 28, 2016

abyravan commented on Feb 28, 2016

soumith commented on Feb 28, 2016

soumith commented on Feb 28, 2016

pvtokmakov commented on Jun 26, 2017

ethanluoyc commented on Aug 15, 2017

sjain-stanford commented on Dec 22, 2017

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions